9.1
Long-run Effects and the Role of Massachusetts’ Temporary Refund
Scheme
The treatment effect of expansion that we measure includes both any competitive effect of charter schools on traditional public schools and the direct effect of a short-term increase in traditional public school funding due to Massachusetts’ refund scheme. Separating these effects is useful both from an external validity perspective (what would we expect in states without refunds?) and a policy perspective (should more states adopt these refunds?).
To provide some evidence on these mechanisms, we analyze whether the spillover effects of expansion persist after districts’ temporary aid ends. The 2011 reform we use above was too recent to measure post-refund effects of expansion. Therefore, we instead exploit, in an event-study framework, the fact that many districts in our sample saw charter openings pre- 2011 which led to large, persistent, and often sudden increases in the share of students attend- ing charter schools. We select large openings similarly to how we chose expanding districts previously, as ones where the district saw faster growth following the opening than before. Specifically, we select openings where the total increase in charter share in the seven years after opening was at least 1 percentage point higher than the increase in charter share from the beginning of our sample period (2002) to the year of opening.47
We then use an event study to estimate the impact of these large openings on districts’ current and future charter share and outcome variables. That is, we estimate the following
47Note that an opening att denotes opening in the school year beginning att, while the outcome variables
are measured for the school year ending int, so an opening attshould only affect the charter share fromt+ 1
equation by OLS: Ydt =α+ k=7 X k=−5 θkOd,t−k+ηd+φt+γdt+it
whereYdt is either the charter share or a financial or achievement outcome variable, andOd,t
denotes that the district had a large opening at timet. We control for district and time fixed effects (ηdandφt) and district-specific linear time trends (γdt). θkis the estimate of the effect
of a charter openingkyears after the event. Note that this specification estimates the effect of charter openings up to seven years after the opening, by which point there would be no further reimbursement due to the initial increase.
The coefficientsθkcannot necessarily be interpreted as causal effects because, as explained
in previous sections, the location and timing of charter openings are likely to be nonrandom.48 However, we have two reasons to believe the results from this event study might be a good guide to longer-run causal effects. Firstly, by including 5 leads of charter opening (k =−1, . . . ,−5), we are able to investigate pre-trends in outcome variables, and perform a Granger test for the direction of causality. Our results show that although the pre-opening coefficient estimates are noisy, there is no clear pre-trend for most outcomes. Secondly, our event study estimates turn out to replicate the synthetic control estimates surprisingly well over the three- to four- year timespan. These two factors give us some confidence that the event study provides useful information on the longer-run effect of charter expansion.
Figure 10 plots the estimated effect of an opening at t on outcomes at t +k for k =
−5, . . . ,7. The effect at t = 0 is normalized to zero. The district charter share rises by about 1.5 percentage points in the year after opening, about 0.5 points in the year after, and subsequently remains largely flat. This is important, as the reimbursement formula depends on theincreasein the number of charter students in past years. Hence, the average treated district should be receiving little refund money byt+ 7.
Over the short term, expanding districts see a large and significant post-opening rise in total per-pupil spending and instructional spending, corresponding well to our IV-SC and IV-DiD results. Also in line with previous results, the effect on fixed costs is largely insignificant, and there appears to be a small positive effect on salaries. Slightly more surprisingly, spending on support services appears to increase, which contradicts our previous results. We tend, however, to trust the causal IV-SC and IV-DiD estimates more.
The long-term effects are directionally similar to the short-term effects, though always smaller. This suggests that the largely positive effects of charter expansion on districts tend to persist, but more modestly, or vanish past the end of the temporary refund.
The effects on achievement are modestly positive in the short run, again corresponding well to our IV-SC and IV-DiD results, and largest over the four- to six-year time horizon. This fits with previous research suggesting that it takes several years for increased spending to impact
48We prefer not to do a synthetic control for these charter openings, as there is no clear way to both adjust for
the fact that different openings happen at different times and simultaneously aggregate outcomes across districts to reduce noise.
achievement (Jackson et al., 2016; Lafortune et al., 2018).
This distinction between the short-term and long-term effects also partially reconciles our results with the negative fiscal spillover effect identified by previous studies, which use data from states where districts are not refunded for their charter tuition expenditures (Arsen and Ni, 2012; Ladd and Singleton, 2018; Cook, 2018). This event study provides suggestive evidence that our main results are driven largely by the temporary injection of extra funding into the school system, rather than any competitive effect. The small long-run effect is also consistent with the idea that the refund effectively “cushions the blow” to districts from a sudden loss of students due to charter expansion.
9.2
Impact of Charter Expansion on the Pupil-Teacher Ratio
We find that the districts facing the largest charter expansion tend to increase their spending on instruction. This might be due to a decreased pupil-teacher ratio in traditional public schools. Indeed, when facing charter school competition, over the short and medium terms, traditional public schools might be losing students without dismissing teachers. We confirm this by com- paring the evolution of the teacher-student ratio in expanding and nonexpanding districts. To do so, we apply the same IV-SC methodology as we have used for the main analysis.
Figure 12 shows our results. The districtwide ratio of students to teachers falls in expanding districts relative to the synthetic control post-reform. Hence class size is also likely to fall. This reduction in class size could be one mechanism for explaining the positive achievement effects we observe, including over the longer run. Numerous studies have demonstrated the positive effect of smaller class sizes on student achievement (Krueger, 1999; Angrist and Lavy, 1999; Hoxby, 2000; Urquiola, 2006; Fredriksson et al., 2013)49and on longer-term outcomes, such as the probability of taking the ACT and SAT exams or being enrolled in college (Krueger and Whitmore, 2001; Chetty et al., 2011). However, according to our placebo inference the effect on the pupil-teacher ratio is not statistically significant.50