There is a strong argument that randomised controlled trials are the ‘gold standard’ of research designs (Torgerson & Torgerson, 2001), yet the notion is a flawed one. While there may be a hierarchy of investigations from a methodological standpoint, there is no such thing as a ‘gold standard’; it should never be considered the route to an investigation free from certain kinds of bias, but rather, the best option available to answer a specific question. The job of the investigator is to select the most appropriate design to enable a question to be answered, rather than simply to select that which is labelled as the ‘best’.
From the agricultural beginnings of the RCT with R.A. Fisher in the early part of the twentieth century - the language of which still resonates in the terminology of the RCT (Torgerson & Torgerson, 2008) - through Lindquist’s (Lindquist, 1940) work in education on avoiding confounding through clustering, the applications of the approach have resonated
71
with social sciences research. It is in the clinical trials of medicine, however, that the RCT has been most used, and with which it has become most commonly associated. That said, the work of such organisations as the Centre for Evaluation and Monitoring30 (CEM) at
Durham University, the Coalition for Evidence-based Education31 at York University and the
What Works Clearinghouse32 in the USA speak to a growing interest in evidence-based
policy and practice in education. In early 2013, the MET Project’s final report (Cantrell & Kane, 2013) was released to great interest; its suggestion that the effectiveness of teachers could be measured came as a result of a large-scale randomised controlled trial (though there is some dispute over the robustness of the claims made in the project (Rothstein & Mathis, 2013)).
The randomised controlled trial offers the best means by which selection bias, regression to the mean and changes over time can be controlled (Torgerson & Torgerson, 2008) by seeking equivalence in both known and unknown variables; the RCT offers the most robust way of controlling for both the observed and unobserved differences in participants.
The calls for greater use of evidence as a basis for decision-making in education have been heard clearly in the UK for well over a decade (Coe, Fitz-Gibbon, & Tymms, 2000). Moreover, advocates of the RCT design have become ever more vocal (Gorard & Cook, 2007; Torgerson & Torgerson, 2001), and to increasingly receptive ears.
The use of randomised controlled trials in the social sciences has, of late, begun to increase in popularity, and the relevance of their findings seen in a new light. Perhaps as a result of a declining global economy, a populace more interested in frugality and value for money,
30http://www.cemcentre.org/ accessed 20/1/13
31http://www.york.ac.uk/iee/coalition.htm accessed 20/1/13 32http://ies.ed.gov/ncee/wwc/ accessed 20/1/13
72
the randomised controlled trial in social sciences seems to be gaining in popularity. Notably, the release of Test, Learn, Adapt (Haynes, Service, Goldacre, & Torgerson, 2012) is suggestive of growing interest in RCTs at the policy level33, both in the UK and other
countries, and one might conjecture that this interest in ascertaining ‘what works’ prior to full implementation at scale has, in part at least, an economic motive.
Furthermore, the creation of the Education Endowment Foundation in 2011 signalled a shift in government thinking towards increased deployment of the RCT design in education contexts. Prompted by action from the Deputy Prime Minister’s Office, the coalition government’s endowment of £125 through the Department for Education34 set out to
address the destructive link between family socio-economic status and educational achievement. Since inception, the EEF has engaged over 1,800 schools in 56 investigations (Collins, 2013), many of which are randomised controlled trials; in June 2014, a survey by the National Foundation for Educational Research (NFER) found that 45% of school leaders were using the Sutton Trust / EEF Toolkit35. According to a presentation given by Sir Kevan
Collins, Chief Executive of the EEF, in November 2015, this figure had risen to 65%36.
Interestingly, the once-adversarial critic of government policy, Dr Ben Goldacre, has, of late, been visible in his support for the UK Department for Education’s approach to policy- making informed by research evidence. Coming from a health science background, Goldacre’s (2013) entry into the debate on the use of evidence in education heralded
33http://www.bbc.co.uk/news/uk-politics-19656595 accessed 16/1/13 34http://educationendowmentfoundation.org.uk/about accessed 20/9/13 35https://educationendowmentfoundation.org.uk/news/45-of-school-leaders-using-the-teaching-and- learning-toolkit/ accessed 12/11/14 36https://twitter.com/gl_assessment/status/662630671450570753 accessed 22/11/15
73
something of the shift in central government thinking. The sight of a popular37 author and
broadcaster sharing a Bethnal Green Academy stage with policy-makers and academics alike made for interesting viewing.
Clearly there is an environment emerging which suggests that there is support for the use of RCTs in education. Publishers of educational resources are beginning to recognise the commercial potential in interventions based on robust evidence, as indicated by Pearson’s publication of a set of materials designed to support writing skills in secondary schools38.
Interestingly, a prominent part of the marketing of this product focuses on the benefits of its basis in research using the RCT design, the findings of which originally appeared in the report on an ESRC-awarded grant (Myhill, Jones, & Bailey, 2011).
Perhaps this is how robust evidence in education will begin to be embedded in schools. Daniel Willingham’s (2012) book (written out of “frustration with a particular problem: the word ‘research’ has become meaningless in education”39) is clearly written with
practitioners in mind, and presents a model for evaluating a proposed intervention. Pearson’s marketing of their writing skills publication follows this model almost entirely to the letter, stating clearly their claims for the intervention’s impact, giving the credentials of the author and the research on which the product is based, highlighting the evidence offered and engaging teachers’ professional desire to raise attainment in a crucial area. This method is nothing new, echoing the structure of rhetoric as described by Aristotle
37 Goldacre had 292,725 followers on his official Twitter account https://twitter.com/bengoldacre accessed 20/9/13 38 http://www.pearsonschoolsandfecolleges.co.uk/Secondary/GlobalPages/SkillsforWriting/SkillsforWriting.as px accessed 20/9/13 39http://www.danielwillingham.com/1/post/2012/07/how-can-you-know-if-its-really-research-based.html accessed 03/04/13
74
(Burnyeat, 1994): ethos, logos, pathos. Perhaps the combination of research, economic imperatives, educational imperatives and rhetoric may offer a fruitful path for embedding evidence in education.
3.4 Design of the Secondary Student Evaluation of Educational Quality randomised