Peaking Interest: How awareness drives the effectiveness of time-of-use electricity pricing

Transcription

1 Peaking Interest: How awareness drives the effectiveness of time-of-use electricity pricing Brian C. Prest 1 Camp Resources XXIV August 7 8, Ph.D. Candidate, UPEP, Duke University, brian.prest@duke.edu 1 Brian C. Prest Peaking Interest

2 Overview and Preview of Results Overview Goal Using machine learning to estimate heterogeneous treatment effects from TOU electricity pricing & info. provision Meaning? Under what conditions do consumers change their electricity consumption in response to prices that vary by time of day? 2 Brian C. Prest Peaking Interest

3 Overview and Preview of Results Overview Goal Using machine learning to estimate heterogeneous treatment effects from TOU electricity pricing & info. provision Meaning? Under what conditions do consumers change their electricity consumption in response to prices that vary by time of day? Heterogeneity Why? targeting, understanding mechanisms Context: rich dataset from experiment on Irish households Problem: multiple testing What s needed: parsimonious way to estimate heterogeneity in many dimensions in a robust manner machine learning 2 Brian C. Prest Peaking Interest

4 Overview and Preview of Results Results Average TOU pricing effect: -9% peak consumption 3 Brian C. Prest Peaking Interest

5 Overview and Preview of Results Results Average TOU pricing effect: -9% peak consumption Robust Sources of Heterogeneity: 3 Brian C. Prest Peaking Interest

6 Overview and Preview of Results Results Average TOU pricing effect: -9% peak consumption Robust Sources of Heterogeneity: Key result: Unaware households don t respond (-2%, insignif.) 3 Brian C. Prest Peaking Interest

7 Overview and Preview of Results Results Average TOU pricing effect: -9% peak consumption Robust Sources of Heterogeneity: Key result: Unaware households don t respond (-2%, insignif.) Low energy consuming households don t respond 3 Brian C. Prest Peaking Interest

8 Overview and Preview of Results Results Average TOU pricing effect: -9% peak consumption Robust Sources of Heterogeneity: Key result: Unaware households don t respond (-2%, insignif.) Low energy consuming households don t respond Info. provision amplifies effects up to 2x (even conditional on awareness) 3 Brian C. Prest Peaking Interest

9 Overview and Preview of Results Results Average TOU pricing effect: -9% peak consumption Robust Sources of Heterogeneity: Key result: Unaware households don t respond (-2%, insignif.) Low energy consuming households don t respond Info. provision amplifies effects up to 2x (even conditional on awareness) Size of price change doesn t matter 3 Brian C. Prest Peaking Interest

10 Overview and Preview of Results Results Average TOU pricing effect: -9% peak consumption Robust Sources of Heterogeneity: Key result: Unaware households don t respond (-2%, insignif.) Low energy consuming households don t respond Info. provision amplifies effects up to 2x (even conditional on awareness) Size of price change doesn t matter Nothing else matters (out of 150+ household characteristics) 3 Brian C. Prest Peaking Interest

11 Overview and Preview of Results Results Average TOU pricing effect: -9% peak consumption Robust Sources of Heterogeneity: Key result: Unaware households don t respond (-2%, insignif.) Low energy consuming households don t respond Info. provision amplifies effects up to 2x (even conditional on awareness) Size of price change doesn t matter Nothing else matters (out of 150+ household characteristics) Other findings: 3 Brian C. Prest Peaking Interest

12 Overview and Preview of Results Results Average TOU pricing effect: -9% peak consumption Robust Sources of Heterogeneity: Key result: Unaware households don t respond (-2%, insignif.) Low energy consuming households don t respond Info. provision amplifies effects up to 2x (even conditional on awareness) Size of price change doesn t matter Nothing else matters (out of 150+ household characteristics) Other findings: Can t reliably predict who will be aware 3 Brian C. Prest Peaking Interest

13 Overview and Preview of Results Results Average TOU pricing effect: -9% peak consumption Robust Sources of Heterogeneity: Key result: Unaware households don t respond (-2%, insignif.) Low energy consuming households don t respond Info. provision amplifies effects up to 2x (even conditional on awareness) Size of price change doesn t matter Nothing else matters (out of 150+ household characteristics) Other findings: Can t reliably predict who will be aware Real-time pricing not necessarily more efficient than TOU 3 Brian C. Prest Peaking Interest

14 Background on Time-of-Use Pricing What is Time-of-Use Pricing? Price ( cents per kwh) Tariff D Tariff C Tariff B Tariff A Control (No Change) Hour of Day 4 Brian C. Prest Peaking Interest

15 Experiment Experimental Details 2009: Households recruited; baseline data collected; pre-trial survey; randomized to treatment/control 2010: Treatment period 2011: Follow-up survey Bi-Monthly Bill / Energy Statement Monthly Bill / Statement In-Home Display Load Reduction Incentive Control Tariff A Tariff B Tariff C Tariff D Control Brian C. Prest Peaking Interest

16 Experiment Experimental Details 2009: Households recruited; baseline data collected; pre-trial survey; randomized to treatment/control 2010: Treatment period 2011: Follow-up survey Price ( cents per kwh) Tariff D Tariff C Tariff B 32 Tariff A Control (No Change) Hour of Day Bi-Monthly Bill / Energy Statement Monthly Bill / Statement In-Home Display Load Reduction Incentive Control Tariff A Tariff B Tariff C Tariff D Control Brian C. Prest Peaking Interest

17 Experiment Experimental Details 2009: Households recruited; baseline data collected; pre-trial survey; randomized to treatment/control 2010: Treatment period 2011: Follow-up survey Price ( cents per kwh) Tariff D Tariff C Tariff B 32 Tariff A Control (No Change) Hour of Day Bi-Monthly Bill / Energy Statement Monthly Bill / Statement In-Home Display Load Reduction Incentive Control Tariff A Tariff B Tariff C Tariff D Control Brian C. Prest Peaking Interest

18 Experiment Data Household Electricity Consumption (N T 77 million) N 3, 000 households T 26, 000 half-hours each 6 Brian C. Prest Peaking Interest

19 Experiment Data Household Electricity Consumption (N T 77 million) N 3, 000 households T 26, 000 half-hours each Survey data: > 150 complete variables on... Family characteristics: Employment status, education, social class, household size, #adults, #children, etc. House characteristics: age, #rooms, #bedrooms, style, insulation, window glazing, etc. Appliance & electronics characteristics: home heating fuel types, water heating fuel types, #immersion heaters, #dishwashers, #washing machines, #tumble dryers, #TVs (by size), #computers, #game consoles, internet access, etc. Attitudinal/Behavioral: Attitudes towards energy, environment, etc., expected achievable energy savings, appliance & electronics usage behavior, reasons for participating in the program, etc. Post-experiment survey: questions about experience with program 6 Brian C. Prest Peaking Interest

20 Results 7 Brian C. Prest Peaking Interest

21 Results 1 Average Treatment Effect 7 Brian C. Prest Peaking Interest

22 Results 1 Average Treatment Effect 2 Heterogeneity 7 Brian C. Prest Peaking Interest

23 Results 1 Average Treatment Effect 2 Heterogeneity 3 Policy Targeting 7 Brian C. Prest Peaking Interest

24 Average Treatment Effect Average Treatment Effect 8 Brian C. Prest Peaking Interest

25 Average Treatment Effect Difference-in-Differences (All Treatments) ln(kwh i,h,t ) = β h T reatment i,h,t + α i,h + λ m,h + ɛ i,h,t 9 Brian C. Prest Peaking Interest

26 Average Treatment Effect Difference-in-Differences (All Treatments) ln(kwh i,h,t ) = β h T reatment i,h,t + α i,h + λ m,h + ɛ i,h,t Treatment Effect (Log Points: ln(yi)) Point Estimate (Insigificant) Point Estimate (Sigificant) Night Rate (9 12 ) Day Rate ( ) Peak Rate (20 38 ) Day Rate ( ) Hour of Day Individual Treatments Pre-trend check 9 Brian C. Prest Peaking Interest Night Rate (9 12 )

27 Heterogeneity 10 Brian C. Prest Peaking Interest

28 Heterogeneity Methods: Athey-Imbens & Extensions Athey & Imbens (2016) Estimates CATEs w/ trees ( Conditional ATE) Extensions in this paper Diff-in-diff (vs. diff ) Replace Y i with Y i Multiple treatment groups Allow splitting on treatment Details 11 Brian C. Prest Peaking Interest

29 Heterogeneity How does Athey-Imbens work? (Simple Version) Y i... outcome, W i... treatment, p... share treated ( n T n ) { Yi W i p Y i p(1 p) = Yi /p if W i = 1 Y i /(1 p) if W i = 0 12 Brian C. Prest Peaking Interest

30 Heterogeneity How does Athey-Imbens work? (Simple Version) 1 n i Y i... outcome, W i... treatment, p... share treated ( n T n ) { Yi W i p Y i p(1 p) = Yi /p if W i = 1 Y i /(1 p) if W i = 0 Y i = 1 n = 1 n T [ i S T = LAT E Y i p + i S C Y i 1 n C i S T Y i 1 p i S C Y i ] = 1 n [ i S T Y i n T /n + i S C Y i n C /n ] 12 Brian C. Prest Peaking Interest

31 Heterogeneity How does Athey-Imbens work? (Simple Version) 1 n i Y i... outcome, W i... treatment, p... share treated ( n T n ) { Yi W i p Y i p(1 p) = Yi /p if W i = 1 Y i /(1 p) if W i = 0 Y i = 1 n = 1 n T [ i S T = LAT E Y i p + i S C Y i 1 n C i S T Y i 1 p i S C Y i ] = 1 n [ i S T Y i n T /n + i S C Y i n C /n ] Intuitively, model Y i as a flexible function of X i : E[Y i X i ] = f(x i ) = AT E(X i ) 12 Brian C. Prest Peaking Interest

32 Heterogeneity Results Heterogeneity Results 13 Brian C. Prest Peaking Interest

33 Heterogeneity Results [4] TE: 14.6% SE: (1.6%) n: 474 [3] TE: 11.2% SE: (1.3%) n: 1933 Info. Treatment: [5] In Home TE: 10.1% Display? SE: (1.3%) n: 1459 [6] TE: 11.7% SE: (1.5%) n: 513 [2] TE: 10.3% SE: (1.4%) n: 2029 Baseline Average Peak Consumption >= 0.12 kwh (5th pctile) Info. Treatment: [7] Monthly TE: 9.3% Bill? SE: (1.4%) n: 946 yes Info. Treatment: Overall Load Reduction Incentive? TE: 10.7% SE: (1.4%) n: 477 [1] TE: 8.9% SE: (0.9%) n: 2328 Aware of Tariff Change? no [8] [9] [10] TE: 7.9% SE: (1.6%) n: 469 TE: 8.4% SE: (12.4%) n: Brian C. Prest Peaking Interest [11] TE: 2.3% SE: (1.7%) n: 299 Baseline Average Peak Consumption >= 0.25 kwh (24th pctile) [12] [13] TE: 4.2% SE: (1.4%) n: 202 TE: 0.2% SE: (4.5%) n: 97 Robustness Checks: Propensity Tree Honest Tree

34 Heterogeneity Results Aside: Mean Demand Curves, by Period of Day Price ( cents per kwh) Peak Hours Day Hours Night Hours Consumption (kwh per 30 minutes) 15 Brian C. Prest Peaking Interest Heterogeneous Demand

35 Heterogeneity Results Aside: Mean Demand Curves, by Period of Day Price ( cents per kwh) Peak Hours Day Hours Night Hours Consumption (kwh per 30 minutes) Commonly-used constant elasticity assumption is invalid Real-time pricing not necessarily better (& possibly worse) 15 Brian C. Prest Peaking Interest Heterogeneous Demand

36 Policy Targeting 16 Brian C. Prest Peaking Interest

37 Who is likely to be aware? Targeting Likely-Aware Households Target on awareness? Why? Costs, customer blowback Classification problem: P (Aware i = 1 X i ) = f(x i ) Same problem: many variables, multiple testing Solution: Post-selection Lasso for variable selection/elimination 17 Brian C. Prest Peaking Interest

38 Who is likely to be aware? Lasso Post-Selection LPMs on Awareness Aware of Tariff Change (Indicator) LPM Lasso-Min Lasso-1SE Internet Access in Home (Indicator) Use Internet Regularly (Indicator) Water Heating Fuel: Oil (Indicator; none omitted) Number of Dishwashers in Home Number of Desktop Computers in Home Expect Participating in Trial Will Reduce My Bill (Indicator) Female Respondent (Indicator) Social Class: AB (Highest) (Indicator) Education: Third (e.g., University) (Indicator) Info. Treatment: In-Home Display (Indicator; Bi-monthly Bill omitted) Info. Treatment: Monthly bill (Indicator; Bi-monthly Bill omitted) Info. Treatment: OLR (Indicator; Bi-monthly Bill omitted) R Adjusted R Observations 2,328 2,328 2,328 Number of Covariates Number of Covariates Not Shown Number of Covariates Significant (5% level) 14 (11.4%) na na 18 Brian C. Prest Peaking Interest

39 Conclusion 1 Awareness of the TOU pricing is key to effectiveness Awareness not predictable, so focus on information 19 Brian C. Prest Peaking Interest

40 Conclusion 1 Awareness of the TOU pricing is key to effectiveness Awareness not predictable, so focus on information 2 Small consumers don t respond (in levels or percentages) Target larger consumers 19 Brian C. Prest Peaking Interest

41 Conclusion 1 Awareness of the TOU pricing is key to effectiveness Awareness not predictable, so focus on information 2 Small consumers don t respond (in levels or percentages) Target larger consumers 3 Information provision amplifies effects up to 2x, even among aware households Information is important 19 Brian C. Prest Peaking Interest

42 Conclusion 1 Awareness of the TOU pricing is key to effectiveness Awareness not predictable, so focus on information 2 Small consumers don t respond (in levels or percentages) Target larger consumers 3 Information provision amplifies effects up to 2x, even among aware households Information is important 4 Size of price change doesn t matter 19 Brian C. Prest Peaking Interest

43 Conclusion 1 Awareness of the TOU pricing is key to effectiveness Awareness not predictable, so focus on information 2 Small consumers don t respond (in levels or percentages) Target larger consumers 3 Information provision amplifies effects up to 2x, even among aware households Information is important 4 Size of price change doesn t matter Getting prices exactly right isn t as important as getting people to pay attention in the first place 19 Brian C. Prest Peaking Interest

44 Thanks! 20 Brian C. Prest Peaking Interest

45 Appendix Balance Tests & Diagnostics Average Consumption Profiles, by Group & Period Consumption (kwh per 30 minutes) Night Rate (9 12 ) Control Group Baseline Period Treatment Period Day Rate ( ) Peak Rate (20 38 ) Day Rate ( ) Night Rate (9 12 ) Hour of Day Balance Checks: t-tests LPM 21 Brian C. Prest Peaking Interest

46 Appendix Balance Tests & Diagnostics Average Consumption Profiles, by Group & Period Consumption (kwh per 30 minutes) Night Rate (9 12 ) Control Group Treatment Group Baseline Period Treatment Period Day Rate ( ) Peak Rate (20 38 ) Day Rate ( ) Night Rate (9 12 ) Hour of Day Balance Checks: t-tests LPM 21 Brian C. Prest Peaking Interest

47 Appendix Balance Tests & Diagnostics Average Consumption Profiles, by Group & Period Consumption (kwh per 30 minutes) Night Rate (9 12 ) Control Group Treatment Group Baseline Period Treatment Period Day Rate ( ) Peak Rate (20 38 ) Day Rate ( ) Night Rate (9 12 ) Hour of Day Balance Checks: t-tests LPM Treatment/control are unbalanced need diff-in-diff 21 Brian C. Prest Peaking Interest

48 Appendix Balance Tests & Diagnostics Balance Checks: t-tests Variable Control Mean Treatment Mean t-statistic p-value Unbalanced Variables (α < 0.05) 1 Employment status: Retired (Indicator) Number of Large Televisions (21+ inch) Number of Electronics Age Group: 65+ (Indicator) Has Children Under 15 in Home (Indicator) Number of Residents Social Class: AB (Highest) (Indicator) Education: Primary only (Indicator) Baseline Average Consumption (Night Hours) Internet Access in Home (Indicator) Number of Desktop Computers Number of Children Under 15 in Home Housing Status: Own with Mortgage (Indicator) Others in Household Use Internet Regularly (Indicator) Selected Balanced Variables (α 0.05) 15 Baseline Average Consumption (Peak Hours) Number of Adults in Home Cook stove type: Electric (Indicator) Number of Laptop Computers Baseline Average Consumption (Day Hours) Unemployed, not seeking job (Indicator) Home Heat: Solid Fuel (Indicator) Interested in changing energy use for environment* Female (Indicator) Education: Secondary to Certificate (Indicator) Satisfied with billing frequency* Observations 3,006 Number of Variables Tested 122 Number of Variables Not Shown 97 Number of Variables Significant (5% level) 14 Share of of Variables Significant (5% level) 11.5% Return to main 22 Brian C. Prest Peaking Interest

49 Appendix Balance Tests & Diagnostics Balance Checks: LPM Dependent variable: Treated (Indicator) Baseline Average Consumption (Peak Hours) 0.03 (0.07) Baseline Average Consumption (Night Hours) 0.14 (0.15) Baseline Average Consumption (Day Hours) (0.12) Number of Large Televisions (21+ inch) 0.02 (0.01) Age Group: 65+ (Indicator) 0.13 (0.11) Number of Adults in Home 0.01 (0.01) Internet Access in Home (Indicator) 0.01 (0.02) Number of Desktop Computers in Home 0.01 (0.02) Others in Household Use Internet Regularly (Indicator) (0.02) Cook stove type: Electric (Indicator) (0.06) Observations 3,006 R Adjusted R Number of Covariates 109 Number of Covariates Not Shown 100 Number of Covariates Significant (5% level) 2 Share of Covariates Significant (5% level) 1.8% Return to main 23 Brian C. Prest Peaking Interest

50 Appendix Balance Tests & Diagnostics Placebo test for TOU pricing: Weekends & Holidays Treatment Effect (Log Points: ln(yi)) Point Estimate (Insigificant) Point Estimate (Sigificant) Night Rate (9 12 ) Day Rate ( ) Hour of Day Peak Rate (20 38 ) Day Rate ( ) Night Rate (9 12 ) 24 Brian C. Prest Peaking Interest

51 Appendix Balance Tests & Diagnostics Diff-in-Diff (Individual Treatments) Tariff A, Bi monthly Bill Tariff A, Monthly Bill Tariff A, Bi monthly Bill + IHD Tariff A, Bi monthly Bill + OLR Tariff B, Bi monthly Bill Tariff B, Monthly Bill Tariff B, Bi monthly Bill + IHD Tariff B, Bi monthly Bill + OLR Treatment Effect Treatment Effect Treatment Effect Tariff C, Bi monthly Bill Hour of Day Tariff D, Bi monthly Bill Hour of Day Treatment Effect Treatment Effect Tariff C, Monthly Bill Hour of Day Tariff D, Monthly Bill Hour of Day Treatment Effect Treatment Effect Tariff C, Bi monthly Bill + IHD Hour of Day Tariff D, Bi monthly Bill + IHD Hour of Day Treatment Effect Treatment Effect Tariff C, Bi monthly Bill + OLR Hour of Day Tariff D, Bi monthly Bill + OLR Hour of Day Brian C. Prest Peaking Interest Return to main

52 Appendix Balance Tests & Diagnostics Checking for pre-trends in peak consumption Consumption (kwh per 30 minutes) Baseline Period Treatment Period Control Group Treatment Group Jul 2009 Sep 2009 Nov 2009 Jan 2010 Mar 2010 May 2010 Jul 2010 Sep 2010 Nov 2010 Time Return to main 26 Brian C. Prest Peaking Interest

53 Appendix Tree Sensitivities Robustness Check: Propensity Tree [1] TE: 8.9% SE: (0.9%) n: 2328 yes Aware of Tariff Change? no [2] TE: 10.4% SE: (1.4%) n: 2029 Baseline Average Peak Consumption >= 0.12 kwh (5th pctile) [3] TE: 11.4% SE: (1.3%) n: 1933 Info. Treatment: In Home Display? [4] [5] [6] [7] TE: 10.3% TE: 8.6% SE: (1.3%) SE: (12.4%) n: 1459 n: 96 TE: 14.7% SE: (1.6%) n: 474 TE: 2.3% SE: (1.7%) n: Brian C. Prest Peaking Interest Return to main

54 Appendix Tree Sensitivities Robustness Check: Honest Tree [1] TE: 8.9% SE: (1.4%) n: 1151 yes Aware of Tariff Change? no [2] TE: 10.1% SE: (2%) n: 607 Baseline Average Peak Consumption >= 0.22 kwh (18th pctile) [3] TE: 9.6% SE: (2%) n: 508 Info. Treatment: In Home Display? [4] [5] [6] [7] TE: 8.6% TE: 8.9% SE: (1.9%) SE: (5.8%) n: 418 n: 99 TE: 12.7% SE: (2.3%) n: 90 TE: 1.4% SE: (2.7%) n: Brian C. Prest Peaking Interest Return to main

55 Appendix Heterogeneous Demand Curves Heterogeneous Demand Curves Price ( cents per kwh) IHD Group, Peak Hours Monthly Bill Group, Peak Hours OLR Group, Peak Hours Bi monthly Bill Group, Peak Hours Unaware, Large User Group, Peak Hours All Households, Day Hours All Households, Night Hours Consumption (kwh per 30 minutes) 29 Brian C. Prest Peaking Interest Return to main

56 Appendix Extension - Multiple Treatment Groups Extension - Multiple Treatment Groups For each control observation i and treatment m {1,..., M}, generate a new pseudo-observation i m with Y im Y i X im X i W im W i ( = 0) { 1 for m For m = m {1,..., M} Wi m m 0 for m m, for a total of M n C control observations, replacing the n C original ones 30 Brian C. Prest Peaking Interest

57 Appendix Extension - Multiple Treatment Groups Extension - Multiple Treatment Groups Transformed LATE: 1 1 Y i n T M n i S C T Y i = 1 i S n T C = 1 n T = ˆτ, Y i i S T Y i 1 n i S C T 1 M M n C i S C Y i i S C Y i which is the same as the LATE estimate of the untransformed data. Return 31 Brian C. Prest Peaking Interest