Quality was assessed using a modified version of The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials.157For many of the items, studies did not report sufficient information to render a
judgement. Findings are presented below separated by the level of the evaluation. Random generation of allocation sequence
Of the 16 included evaluations, 10 evaluations, reported on in 21 publications, did not report enough information to judge the risk of bias arising from allocation sequence generation.60,87,88,90–93,95–97,103,104,109–111, 128–130,132,134,136The remaining six evaluations (4Rs,3,52,53KAT,127LIFT,98,100,125Second Step,139–141Learning to
Read in a Healing Classroom137,138and Positive Action Chicago83,113–116) all presented enough information
to appraise these evaluations as having a low risk of bias in this domain. Both 4Rs52and Positive Action
Chicago115,116used random number generators, whereas LIFT drew allocations from a hat.125KAT used
optimal allocation to determine the randomisation sequence.127
Allocation concealment
In total, 15 out of 16 included evaluations did not state if or how allocation was concealed, thus preventing the determination of the risk of bias in this domain. Learning to Read in a Healing Classroom137,138was
assessed as having a low risk of bias because of reported methods of concealment. Blinding
Although blinding is often difficult in trials of school interventions, blinding of outcome assessors may often be possible and is useful for reducing ascertainment bias. Of the 16 included evaluations, only one provided enough information to judge whether or not any blinding occurred. In LIFT,98,125outcome
Complete outcome data
In total, 15 out of 16 evaluations were judged as having a relatively low or balanced attrition when this was relevant to outcome assessment. In some evaluations, such as the evaluations of Positive Action,87,113–116
whole-school intervention implementation meant that students were followed up at a certain grade level regardless of the duration of their time in the school or whether or not baseline data were collected. Learning to Read in a Healing Classroom137,138was assessed as having a high risk of bias because of the high
levels of missing data. Complete reporting
Of the 16 evaluations, 12 were judged as being at an unclear risk of selective reporting. This was generally due to a lack of a trial protocol. Two evaluations, 4Rs51and Second Step,139–141were judged as being at a
high risk of bias from selective reporting because of unclear and incomplete reporting of findings from the 3-year results. Bullying Literature Project–Moral Disengagement and Learning to Read in a Healing Classroom were both judged as being at a low risk of bias because all expected outcomes were reported. Confounding addressed
Although randomisation should on average produce groups balanced for known and unknown confounders, one of the issues arising in school-randomised or classroom-randomised trials is that the relatively small number of higher-level units (i.e. schools) can produce imbalances between arms. Of the 16 evaluations, 10 were at a low risk of bias from confounding. In these evaluations, confounding was addressed by including a vector of person-level confounders in multivariate regression models. The remaining six evaluations were at an unclear risk of confounding because inadequate information was presented in outcome evaluations.
Accounting for clustering
Evaluations generally were at a low risk of bias owing to inadequate assessment of clustering because clustering was generally well assessed. Of the 16 evaluations, 13 (reported in 35 publications) used appropriate analytical methods to account for clustering;3,51–53,60,83,87,88,90,91,93,97,98,100,101,103,104,109–111,113–116,125, 127–130,132,137–141for example, robust standard errors in the Gatehouse Project90,91or generalised linear
mixed-effects models to nest students within schools (e.g. 4Rs, Positive Action Hawaii and Chicago, PATHS and Youth Matters3,51–53,83,87,109–111,113,114,116,129,130). One evaluation, I-LST,97,115,128,132presented results
that did not account for clustering but observed that results accounting for clustering were not different. One evaluation had an unclear risk of bias (RHC92,95) and two had a high risk of bias in this domain
(Bullying Literature Project96,134and Bullying Literature Project–Moral Disengagement136).
Additional sources of bias
Most evaluations (14/16, reported on in 35 publications)3,51–53,60,83,87,88,90–93,95,97,98,100,101,103,104,113–116,125,127–130,132
used a variety of methods to reduce other sources of bias; for example, additional methods of addressing missing data or sensitivity analyses for different model assumptions. Two evaluations, Youth Matters110,111
and Bullying Literature Project,96,134did not present the relevant information to permit a clear determination
of other sources of bias. Suitable control group
Of the 16 evaluations, 10 (reported in 29 publications) reported enough information to judge that the intervention and control groups were similar on key characteristics or accounted for differences in the analysis.3,51–53,60,83,87,88,90–92,95,109–111,113–116,128–130,132,136–141However, six evaluations (reported in 12 publications)
did not provide enough information to judge whether or not the intervention and control groups were sufficiently similar; the evaluations did provide evidence of baseline differences between the groups but did not demonstrate how these differences were accounted for.93,96–98,100,101,103,104,125,127,131,134
Grading of Recommendations, Assessment, Development and Evaluations assessment of primary outcomes
The quality of evidence for outcomes by KS is presented in Table 3. For most combinations of outcome and KS, the quality of evidence was low or very low. However, for substance use outcomes in KS3 and 4 and for violence victimisation outcomes in KS2, quality of evidence was moderate.