Customers in the experiment were randomly assigned to one of three mutually exclusive groups:
(i) Control: customers who never receive any correspondence from Opower; (ii) HER Only:
customers who only receive a monthly HER; and (iii) HER+Rewards: customers who receive a
monthly HER and are encouraged to enroll in the rewards program.
We would have preferred to implement a 2x2 experimental design with an additional group
that only provides financial rewards. However, as take-up rates in past voluntary programs were
low, the partner utility was concerned that such a treatment would not generate substantial changes
in electricity use. Given these concerns and a limited sample size, we decided to focus on three
treatment cells and eschew a rewards-only group.
We summarize the outcomes of the randomization in Figure 11. Unlike the standard admin-
istration of the HER, the financial rewards program is designed as an opt-in policy. We exploit
this design feature to achieve four goals: (i) derive a clean measure of how the HER affects us-
age; (ii) understand how these responses are affected by the introduction of the rewards program;
(iii) study the customer types that choose to participate in the new program; and (iv) evaluate the
Figure 11: Experimental Design and Treatment Assignment Households (N =195,826) HER+Rewards (N =149,997) HER Only (N = 28,061) Control (N =17,768)
Notes:Customers are randomly assigned to one of three mutually exclusive treatment groups. Control customers do not receive any correspondence from Opower. HER Only customers re- ceive monthly HERs beginning in March 2013. HER+Rewards customers are encouraged to participate in the rewards program in addition to receiving monthly HERs. Ndepicts the sample size within each group. Due to administrative constraints, the experiment was implemented in two deployment waves with identical treatments and randomization procedures. For evidence of a successful randomization, please consult Table B1 and Table B2 in the appendix.
To encourage participation, customers in the HER+Rewards group received encouragement
messages over the course of four months. Common to all encouragements is the offer of an ini-
tial balance worth 200 points ($2) for completing the enrollment process.54 In the first month,
the encouragement message was featured on the front page of the HER. The message highlighted
the new financial rewards program and provided information about the sign-up process. Figure
12 presents an example. Except for the inclusion of the encouragement message, the HER was
identical to the reports received by customers in the HER Only group.55 In the following three
months, customers in the HER+Rewards group who had not yet signed up for the financial re-
wards program received a monthly email that contained the same encouragement message. These
messages were purely informational and specific to the financial rewards program—customers
in other groups were not made aware of the program. The messages also do not introduce other
energy conservation campaigns, raise awareness of the HER and strategies to conserve electricity,
or include normative appeals for energy savings.
54The average monthly rewards points for reductions in usage are approximately 60. Hence, our signup bonus is
equal to two and a half months of savings, on average.
55Opower routinely uses similar marketing materials in their HERs to encourage the uptake of utility programs,
such as incentives for technology adoption. Thus, customers are used to seeing slightly different versions of the HER over time.
Figure 12: The Encouragement Message
Notes:Example encouragement module included in the May 2013 HER for customers in the
HER+Rewards group. Customers also received emails containing the same content in June, July, and August 2013.
As part of the encouragement design, we randomized customers into four subgroups that re-
ceived different versions of the marketing messages. These versions varied in terms of how we
framed the sign-up bonus and program participation (a similar approach is used in Saez, 2009;
Bertrand et al., 2010; Hahn et al., 2016). For example, in one group we worded the sign-up bonus
as a loss instead of a gain. Across all four framings, we included the sentence “Earn points for
every kWh you save and get rewarded” and presented a sample of goods that customers can pur-
chase with rewards points. All modules also included the web address of the sign-up website. For
this study, we pool all customers who signed up for the rewards program irrespective of when
they signed up or which behavioral framing they faced. We refer to this subset of customers as
participantsthroughout the paper.
Figure 13 summarizes the different elements of the experiment and its timeline. Because of
administrative constraints, the study was implemented identically in two deployment waves, i.e.,
two separate experiments. The two experiments only differ regarding pre-intervention average
usage and household characteristics. Opower randomized customers within each deployment
wave, and we include fixed effects to control for baseline differences across the experiments in
Figure 13: Timeline of the Experiment Mar2012 Apr2015 Mar2013 Home Energy Reports May2013 Rewards Program Sep2013 End of Encouragement
Notes:The general timeline of the experiment. Vertical lines represent the begin dates of impor- tant interventions and rectangles represent the duration. We observe one year of energy usage before the first HER in March 2013. The marketing module for the rewards program was in- cluded in the May 2013 HER and encouragement emails were delivered from June to August 2013. We observe average daily usage for each month until April 2015. The timing and content of all interventions were identical in both deployment waves of the HER.
We begin observing consumption data in March 2012.56 In March 2013, customers in the
HER Only and HER+Rewards groups received the first HER. After two months of regular HERs,
customers in the HER+Rewards group received the encouragement message contained in the
third HER in May 2013. In the following three months, June, July, and August 2013, Opower
delivered encouragement emails to non-participating households in the same group.