Respuesta de la vegetación 64

3.1 Chapter Introduction

This section reviews the current use of value-added in educational effectiveness research, English policy and English practice. This draws on the technical details presented in Chapter 2 concerning the purposes, designs and outputs of value-added in order to discuss the application of the value-added method in each area of use. This chapter is organised into four main sections. The first two consider the use of value-added in educational effectiveness research (Sections 3.2 and 3.3), the last two consider the use of value-added in policy (Section 3.4) and practice (Section 3.5), respectively.

This chapter provides important foundations for the discussion and conclusion chapters which explore the implications of the new and pre-existing evidence presented in this thesis for the validity of value-added in relation to different areas of use.

3.2 Survey of Educational Effectiveness

Research Use of Value-Added

3.2.1 Methodological Survey

Chapter 2 (Section 2.3) outlined a broad conception of the value-added method which had many different formulations and several possible outputs. These were linked by way of their common correlational, production function approach to identifying educational effectiveness factors. Elsewhere, this has been described as an econometric approach (Marsh et al., 2011). Identifying school effects and producing school value-added measures is one possible application of the value-added method. Section 3.2.1 reviews the scale and purposes of various uses of value-added in current educational effectiveness research (EER). This is done through a survey of the two highest ranked educational effectiveness journals (Scimago Lab, 2016), taken to be representative of current research methodology in EER: namely, School Effectiveness and School Improvement (SESI) and the Journal of Research on Educational Effectiveness (JREE). While neither EER nor the use of the value-added method

in research is confined solely to these journals, it was preferred to review use in a core area (which may be expected to be following best practice) in detail than to review use across all educational research at a lower level of detail. The two journals reviewed are held to share similar goals and make use of fairly similar methods to EER published elsewhere in terms of variety but maybe not entirely in terms of proportion. Note that the term EER and other terms such as educational effectiveness and improvement (EEI) tend to be used in self- reference by researchers associated with the SESI journal and with the two handbooks reviewing the field (Teddlie and Reynolds, 2000, Chapman et al., 2015). However, for present purposes, EER is used in a general sense to refer to all sub-groups in what is an increasingly inter-connected and international field (see Chapman et al., 2015, pp.1-4, for an overview).

The following survey was of all papers published in SESI and JREE from January 2013 to when the survey was conducted in mid-April 2015. All papers were collected, read and the key methodological details of the publications were recorded through a simple sorting process (see below). In total, 9 issues of SESI and 10 issues of JREE had been published in this period. Only articles were considered and all content listed as commentaries, corrigendum, miscellany, introductions, editorials or notes were excluded from the survey. In total, 100 articles were surveyed: 44 from JREE, 56 from SESI. These articles were sorted into 7 categories according to the main methods used in the paper. The categories were designed to shed light on the use of value-added rather than give general readers an overview of methods used in the field. Two distinctions introduced in Chapter 2 were used to create the four categories for the correlational methods related to the value- added method: first, whether the study concerns longitudinal outcomes, as described in the growth models section (Chapter 2, Section 2.3.5) or cross-sectional outcomes, as elsewhere (Chapter 2, Section 2.3.2 to 2.3.4). Second, a distinction was made between studies which were primarily interested in the fixed effects from statistical models and those primarily concerned with the latent random effects or residuals (see Chapter 2, Section 2.3.4). Studies which fall outside of correlational methodologies were grouped into three groups: the first group was for experimental and quasi-experimental approaches. These are the key methodological alternative to value-added analysis, controlling for pupil differences by design rather than statistical analysis. The second group contained studies dealing with qualitative evidence or the implementation and refinement of research instruments (e.g.

through factor analysis or item response analyses). Third, there was a group for studies which reflected on the field or knowledge-base through theory, review or simulation studies. The second and third groups were far more general and are not considered in detail because these methods were not the main concern of the survey.

Several of the 100 articles showed more or less equal concern with more than one type of analysis and, as a result, are placed in two categories, bringing the total to 107. For simplicity, however, all figures are referred to as representing single papers. The results are given in Table 3.2.1a, below:

Table 3.2.1a – Methods used in Educational Effectiveness Research

Method SESI JREE Total

Cross-sectional outcome, correlational, random

effects focus 4 1 5

Cross-sectional outcome, correlational, fixed

effects focus 14 0 14

Longitudinal outcome, correlational, random

effects focus 7 1 8

Longitudinal outcome, correlational, fixed effects

focus 16 5 21

Experimental or quasi-experimental 1 20 21

Qualitative analysis, implementation or

methodological study 10 1 11

Theory, simulation or review 10 17 27

Total 62 45 107