Método: tapas y paredes
7. Figura: pantalón
Analyses in the last section have shown that the academic effectiveness of the primary school predicts students’ attainment in KS3 (particularly in maths and science) over and above that attributed to students’ background. It is therefore important to establish whether secondary school effectiveness and educational quality also help to predict better student outcomes at age 14. In order to do this, national data sets have been used to obtain indicators of the level of secondary schools academic effectiveness and quality.
The measures of secondary school academic effectiveness and quality were added to the contextualised models that predicted attainment in Year 9 English, maths and science when controlling for individual student, family and HLE characteristics (see Figure 3.28 for the statistical analysis strategy).
Figure 3.28: Strategy of Statistical Analysis of Net Secondary School Effects
Individual
Pupil
Factors
Family
Factors
Home
Learning
Environment
ENGLISH
MATHEMATICS
SCIENCE
Secondary
School
91
3.3.1. The Impact of Secondary School Academic Effectiveness on Year 9 Attainment
The secondary school academic overall effectiveness was represented by the contextual value added (CVA) score at the school level. This measure25 was provided by the DfE26 and was matched into our data set based on the school identification number.
A mean CVA score was calculated based on KS2 to KS4 (KS2-4) CVA scores for four years from 2006 to 2009 for the secondary schools attended by EPPSE students. This measure was then incorporated into the multilevel models that predicted attainment in Year 9 English, Maths and Science when controlling for individual student, family and HLE characteristics.
We did not find a statistically significant overall effect of the secondary school academic effectiveness for any of the academic outcomes. It has to be noted that the secondary school academic effectiveness is a measure that was used as an overall progress across 5 years and did not relate to the KS3 period. Also, this measure might not relate in the same way with specific academic outcomes as a subject specific academic effectiveness (our earlier analyses on subject specific academic effectiveness measures at primary school indicated that this is relevant).
3.3.2. The Impact of Secondary School Quality on Year 9 Attainment
The quality of secondary schools was measured by Ofsted school level inspection judgements. These judgements cover four dimensions at the school level:
a) overall effectiveness,
b) achievement and standards,
c) personal development and well-being d) quality of provision.
Secondary schools were given grades from 1 to 4, where Grade 1 meant that the secondary school was outstanding, Grade 2 – secondary school was good, Grade 3 – secondary school was satisfactory and Grade 4 – meaning that the secondary school was inadequate.
Since secondary schools are inspected in different years, we collected Ofsted inspection judgements from 2005 until 201027. When a secondary school had several Ofsted inspection judgements, we considered the earliest one in time. The judgements pertaining to the four dimensions mentioned were tested in the contextualised models that predicted academic attainment in Year 9, controlling for individual, family and HLE characteristics (see Appendix 9 for an exhaustive list of the tested Ofsted inspection judgements). Each inspection judgement was entered separately in the model so we avoid potential collinearity problems.
25
At the student level, the CVA score was calculated as the difference between predicted attainment (i.e., the average attainment achieved by similar students) and real attainment in KS4. The predicted attainment was obtained by using multi-level modelling when controlling for students’ prior attainment and adjusting for their background characteristics (i.e., gender, age, ethnicity, special educational needs, FSM, mobility etc.). For each school, all individual student scores were averaged and adjusted for the proportion of students attending the school in a specific year. This final averaged score represents the school level CVA and it is presented as a number based around 1000 (for more technical details see
http://www.education.gov.uk/performancetables/schools_08/2007_2008_Guide_to_CVA.pdf).
26
However, DfE no longer uses this approach. A value added measure is used instead which compares progress, but does not take background into account. The pupil's value added score is based on comparing their exam performance with the median exam performance of other pupils with the same or similar prior attainment at Key Stage 2. The median value is the middle value - with half of the pupils having a capped point score at or below the median, and half at or above. A school's value added measure is a simple average (arithmetic mean) of the value added scores for all pupils in the school.
http://www.education.gov.uk/performancetables/schools_05/sec9.shtml 27
92
Only two Ofsted inspection judgements proved to be significant predictors of students’ academic attainment in Year 9: the ‘quality of pupils' learning and their progress’ (pertaining to the ‘achievement and standards’ dimension) and the ‘attendance of learners’ (part of the ‘personal development and well-being’ dimension).
The Impact of the Quality of Pupils’ Learning and Their Progress on Year 9 Attainment
Students attending secondary schools classified as outstanding based on the quality of pupils’ learning had significantly better average results in English (ESOrig=0.42; ESImputed=0.41), maths
(ESOrig=0.56; ESImputed=0.57) and science TA levels (ESOrig=0.51; ESImputed=0.54) than students from
secondary schools characterised as inadequate in their learning quality.
Additionally, students from secondary schools characterised as good or even satisfactory on learning quality performed significantly better in maths than students from inadequate secondary schools (see Table 3.39).
It is not surprising to find a strong relationship between the overall quality of learning in a specific secondary school and learning outcomes of its students; an outstanding provision leading to higher levels of attainment.
Table 3.38: Contextualised Models for English Teacher Assessment Levels in Year 9: Ofsted Judgements (Original Data vs. Imputed Data)
Year 9 English TA Original Data
Year 9 English TA Imputed Data STATA ICE
Number of students 2463 2996
Number of schools 533 799
Fixed Effects Coef SE ES Sig Coef SE ES Sig
Ofsted Judgement- The Quality of Pupils’ Learning (compared to inadequate)
Outstanding 0.34 0.11 0.42 * 0.35 0.12 0.41 * Good 0.04 0.09 0.05 0.03 0.10 0.04 Satisfactory 0.07 0.08 0.08 0.05 0.09 0.06 Missing 0.08 0.12 0.10 -0.18 0.10 -0.21
% Reduction school variance 81% 71%
% Reduction student variance 25% 17%
% Reduction total variance 38% 31% * p <0.05
Table 3.39: Contextualised Models for Maths Teacher Assessment Levels in Year 9: Ofsted Judgements (Original Data vs. Imputed Data)
Year 9 Maths TA Original Data
Year 9 Maths TA Imputed Data STATA ICE
Number of students 2500 2996
Number of schools 536 799
Fixed Effects Coef SE ES Sig Coef SE ES Sig
Ofsted Judgement- The Quality of Pupils’ Learning (compared to inadequate)
Outstanding 0.64 0.14 0.56 * 0.67 0.14 0.57 * Good 0.29 0.11 0.26 * 0.28 0.12 0.24 * Satisfactory 0.25 0.11 0.22 * 0.23 0.11 0.20 *
Missing 0.43 0.16 0.37 * 0.12 0.12 0.10
% Reduction school variance 85% 82%
% Reduction student variance 17% 12%
% Reduction total variance 30% 27% * p <0.05
93
Table 3.40: Contextualised Models for Science Teacher Assessment Levels in Year 9: Ofsted Judgements (Original Data vs. Imputed Data)
Year 9 Science TA Original Data
Year 9 Science TA Imputed Data STATA ICE
Number of students 2465 2996
Number of schools 534 799
Fixed Effects Coef SE ES Sig Coef SE ES Sig
Ofsted Judgement- The Quality of Pupils’ Learning (compared to inadequate)
Outstanding 0.46 0.11 0.51 * 0.51 0.12 0.54 * Good 0.15 0.09 0.16 0.17 0.10 0.18 Satisfactory 0.06 0.08 0.07 0.07 0.09 0.07 Missing 0.17 0.12 0.19 -0.10 0.10 -0.11
% Reduction school variance 89% 83%
% Reduction student variance 17% 10%
% Reduction total variance 34% 27% * p <0.05
The Impact of the Learners’ Attendance on Year 9 Attainment
Ofsted inspectors rated secondary schools based on the level of attendance of their students. Learners’ attendance as rated by Ofsted inspectors was a statistically significant predictor of academic attainment in Year 9 in all three subjects. Students from secondary schools rated as outstanding on the learners’ attendance got higher average levels in English (ESOrig=0.70;
ESImputed=0.64), maths (ESOrig=0.71; ESImputed=0.69) and science TA (ESOrig=0.56; ESImputed=0.54)
than students from secondary schools characterised as inadequate in their overall attendance (see Table 3.41 - Table 3.43).
Additionally, students from secondary schools characterised as good or even satisfactory on attendance performed significantly better in English, maths and science than students from inadequate secondary schools.
The relationship between overall attendance and academic outcomes is interesting, but not straightforward. The positive strong relationship might be due to the fact that secondary schools that offer high quality academic provision have stricter policies on attendance and therefore are better attended, or are better attended because of the very quality of the provision. Additionally, we could not dismiss the benefits of higher levels of attendance on academic attainment. At this point, we could only speculate about these complex relationships.
Table 3.41: Contextualised Models for English Teacher Assessment Levels in Year 9: Ofsted Judgements (Original Data vs. Imputed Data)
Year 9 English TA Original Data
Year 9 English TA Imputed Data STATA ICE
Number of students 2463 2996
Number of schools 533 799
Fixed Effects Coef SE ES Sig Coef SE ES Sig
Ofsted Judgement- The Attendance of Learners
(compared to inadequate)
Outstanding 0.57 0.11 0.70 * 0.55 0.13 0.64 * Good 0.44 0.10 0.53 * 0.39 0.11 0.45 * Satisfactory 0.43 0.10 0.52 * 0.40 0.11 0.46 *
Missing 0.42 0.12 0.51 * 0.14 0.11 0.16
% Reduction school variance 83% 73%
% Reduction student variance 25% 17%
% Reduction total variance 39% 31% * p <0.05
94
Table 3.42: Contextualised Models for Maths Teacher Assessment Levels in Year 9: Ofsted Judgements (Original Data vs. Imputed Data)
Year 9 Maths TA Original Data
Year 9 Maths TA Imputed Data STATA ICE
Number of students 2500 2996
Number of schools 536 799
Fixed Effects Coef SE ES Sig Coef SE ES Sig
Ofsted Judgement- The Attendance of Learners
(compared to inadequate)
Outstanding 0.82 0.14 0.71 * 0.82 0.15 0.69 * Good 0.60 0.12 0.52 * 0.59 0.13 0.50 * Satisfactory 0.48 0.12 0.42 * 0.47 0.12 0.40 * Missing 0.65 0.15 0.56 * 0.37 0.14 0.32 *
% Reduction school variance 90% 88%
% Reduction student variance 17% 11%
% Reduction total variance 30% 27% * p <0.05
Table 3.43: Contextualised Models for Science Teacher Assessment Levels in Year 9: Ofsted Judgements (Original Data vs. Imputed Data)
Year 9 Science TA Original Data
Year 9 Science TA Imputed Data STATA ICE
Number of students 2465 2996
Number of schools 534 799
Fixed Effects Coef SE ES Sig Coef SE ES Sig
Ofsted Judgement- The Attendance of Learners
(compared to inadequate)
Outstanding 0.51 0.11 0.56 * 0.51 0.13 0.54 * Good 0.36 0.10 0.40 * 0.36 0.11 0.38 * Satisfactory 0.26 0.10 0.28 * 0.26 0.10 0.27 *
Missing 0.33 0.13 0.36 * 0.07 0.11 0.07
% Reduction school variance 91% 85%
% Reduction student variance 17% 10%
% Reduction total variance 34% 27% * p <0.05
3.3.3. The Combined Impact of Pre-School Experience and Secondary School
Academic Effectiveness
Even though we did not find a statistically significant overall effect of the secondary school academic effectiveness, it was important to explore whether features of the secondary school in combination with certain characteristics of the pre-school or primary school would have any influences on academic attainment in Year 9. We wanted to establish whether going to a high quality or more effective pre-school had a protective influence for students who later went to a less effective secondary school, and whether ‘home’ students, or those who went to a less effective or low quality pre-school, did better later if they went to a more effective secondary school.
Therefore, we combined different pre-school measures (i.e., quality and effectiveness) and secondary school academic effectiveness and incorporated them in the same model - controlling for background factors - to explore any joint effects of pre-school and secondary school. For all three subjects, the reference group was ‘no pre-school and low academically effective secondary school’. The results of the combined effects between pre-school quality and secondary school academic effectiveness are presented in Table A.7.1 - Table A.7.3 in Appendix 7.
The benefits of pre-school quality for later attainment in English, maths and science are only visible only for those who were attending a medium effective secondary school in Year 9. The results for the other two groups (low and high effective secondary schools) are harder to interpret. The levels of attainment for this ‘middle’ group are gradated based on the quality of the attended pre-school. Thus, for all three subjects, students who had attended a low, medium or high quality pre-school
95
showed better attainment than those who had never attended a pre-school (given that they were all in a medium effective secondary school (see Figure A.7.1- Figure A.7.3 in Appendix 7).
The combined term between pre-school effectiveness and secondary school academic effectiveness show a similar pattern as the combined term between pre-school quality and secondary school effectiveness. However, the gradation in attainment based on pre-school effectiveness was visible not only for the medium effective secondary schools but also for the low effective secondary schools (see Figure A.7.4 and Figure A.7.6 in Appendix 7).
For all three subjects, the progression in attainment from no pre-school to highly effective pre- schools lead to increase in outcome in low effective secondary schools, students having attended a highly effective pre-school receiving the highest attainment levels when compared to students who had not attended any pre-school.
Similarly, this was the case for medium effective secondary schools. The differences in attainment from no pre-school and low secondary school effectiveness increased as the level of pre-school effectiveness improved, with the largest difference being for students who had attended highly effective pre-schools.