La Teoría del Desarrollo

There is relatively little past research on the attainment impacts of charter schools. One study examining impacts in Chicago and Florida suggests that charter schools may have positive effects on both high school graduation and college attendance (Booker et al. 2011). Since we have only a limited sample of CMOs with the relevant data available, we are not able to provide comprehensive assessment of the overall average CMO impact on these attainment outcomes. Even so, the average impacts for the CMOs with available data are a useful indicator of the efficacy of some CMOs.

The average impacts of these CMOs on the attainment outcomes are positive but not statistically significant. As shown in Table IV.5, the average CMO in our sample has a 7 percentage point impact on high school graduation, but the effect is not statistically significant at the five percent level. For college entry, on average we find a positive impact of 13 percentage points, but the mean impact is not significant (p=0.10) for the limited number of CMOs in the sample. These average impacts correspond reasonably well with those reported in the study of Chicago and Florida charter schools, which found a 7 to 15 percentage point impact on high school graduation and an 8 to 10 percentage point impact on college attendance (Booker et al. 2011).

We have ninth grade high school test score impact estimates alongside high school attainment impacts for a small subset of three CMOs. For these three CMOs, impacts on test scores do not

IV. CMO Schools’ Impacts on Students

impacts for some charter high schools and voucher programs (Booker et al. 2011; Wolf et al. 2010). Although we cannot draw strong conclusions from a sample of three CMOs, the discrepancies in test-score impacts and attainment impacts in this small sample suggests the potential importance of examining attainment outcomes alongside test scores in future studies, to obtain a more complete picture of charter schools’ long-term effects on students.

V. Structures and Practices Associated with Student Impacts

V. STRUCTURES AND PRACTICES ASSOCIATED WITH STUDENT IMPACTS

A. Introduction

Might the variation in the practices of CMOs explain why CMO impacts vary substantially? To inform policy and practice, we examine the associations between CMO practices and impacts, with the aim of identifying practices that are associated with larger positive achievement effects on students. We also explore whether other factors, including CMO size and growth and state charter policies, are associated with impacts.

The results in this chapter are best considered exploratory. Most of the analysis is based on bivariate associations between a single CMO-level practice and student impacts in math or reading. Although we also conduct some multivariate regression analyses of the relationship between impacts and several practices, multicollinearity and limited sample sizes impede our ability to parse out the importance of each practice based on its association with impacts. And any observed associations between practices and impacts could be driven by other, unmeasured factors that are correlated with both the practice and impacts. Hence it is not possible to make causal inferences on the basis of these analyses.

The analysis is conducted at the CMO level, reflecting our primary interest in the policies and practices of CMOs rather than of individual schools. The analysis covers the six primary hypotheses, discussed in Chapter III, of associations between CMO practices and impacts. Appendix Table K.1 shows CMO-specific values for each of these measures, along with estimated impacts and baseline test scores. We also explore a longer list of 43 secondary hypotheses that relate CMO policies and practices to student impacts. These 43 secondary hypotheses include proposed mediators of the association between our primary hypotheses measures and impacts, alternative measures of our primary hypotheses, and other practices not captured by our primary hypotheses. All of the primary hypotheses and nearly all of the secondary hypotheses were defined before conducting the impact analysis. We performed multiple comparison adjustments for our primary hypotheses to test the robustness of our main results accounting for multiple tests.

Key Findings

• Among CMOs, school-wide behavior policies and intensive coaching of new teachers are positively associated with student impacts in both math and reading.

• At the CMO level, we do not find impacts to be associated with use of a uniform curriculum, extended instructional hours, frequent formative student assessment, or performance-based compensation.

• Intensive teacher coaching in CMOs may increase student achievement in part by increasing the frequency with which teachers modify their lesson plans using the results of student assessments.

Results for secondary hypotheses should be considered especially exploratory, given the large number of them. We provide complete results in Appendix Table K.2, Appendix Table K.3, and Appendix Table K.4 for all secondary hypotheses, but in the main text of this chapter, we focus predominantly on those secondary hypotheses can shed light on the findings related to primary hypotheses. For secondary hypotheses, we do not adjust for multiple comparisons.

In the remainder of the chapter, we first describe the methods and data in more detail. We begin our discussion of the results by presenting the associations between each primary hypothesis measure and impacts in math and reading. Next we summarize how impacts are related to the main primary hypotheses. We then elaborate findings related to the three primary hypothesis measures that are significantly associated with student impacts: school-wide behavioral policies, intensive teacher coaching, and extended instructional time. The discussion of the teacher coaching findings also includes a summary of our analysis of the interrelationship among coaching, measures of instructional coherence, and impacts. We then summarize our findings related to a number of staffing practices, including the use of Teach For America (TFA) and Teaching Fellow teachers. Finally, we present average impacts for the categories of CMOs discussed in Chapter III, including those defined by the primary hypotheses and by the prescriptiveness of CMOs. The methods employed in the chapter are summarized in Appendix J and the detailed results from all the analyses are included in Appendix K.