PROVINCIA DE PALENCIA

The methods were feasible and acceptable, with all schools remaining in the study and extremely high

student response rates (> 93% at baseline and follow-up), but further refinements are, nonetheless,

suggested based on the learning from the pilot. Primary outcomes

The primary outcomes investigated within a Phase III trial should include one measure of bullying

victimisation and one measure of the perpetration of aggressive behaviours. Our GBS and ESYTC measures performed satisfactorily and should therefore constitute these two outcomes, respectively. The trial should be powered on the basis of these two outcomes.

Secondary outcomes

The secondary outcomes investigated in a Phase III trial would include all those hypothesised for the pilot as well as validated measures of drug use and sexual risk behaviour (age at sexual debut; and contraception use at last sex). We will also seek to measure educational attainment, as the intervention is hypothesised to have demonstrable effects in this area, and this would probably be powerful evidence in enabling the scaling up of the intervention.

Sample of schools

The sample of schools in a Phase III trial should continue to be diverse but should reflect the overall population profile of schools in the study area (south-east England), rather than aiming to oversample particularly challenging schools (as the pilot did to ensure a diverse range of contexts were included). This pilot trial was initiated in July 2011, 3 months after originally planned, which seriously impeded our ability to recruit schools, although this was nonetheless completed.

Recruitment of schools for a Phase III trial will require a longer lead-in period. We recommend that:

l the project be initiated in February to enable liaison with schools to proceed for 4–6 months before the

summer holidays

l the trial team partners with existing networks of schools such as the Institute of Education’s ‘Teaching

Schools’ network and other school practice networks, such as UCL Partners schools network and

‘Challenge Partners’ (www.challengepartners.org)

l comparison schools continue to be offered £500 in total to cover the expenses for data collection

as well as a report of information from baseline and follow-up surveys (once the trial has been completed).

Surveys

Surveys (of students and teachers) should be conducted in the summer term each year, with baseline surveys undertaken in the summer term prior to the school year in which the intervention occurs. Additionally we recommend that:

l information is provided to students and staff in more accessible language, including about how

anonymity is maintained within the trial

l a Phase III trial should include baseline and follow-up surveys with teachers but also with teaching

assistants and other school staff

l staff surveys should also be conducted in the summer term prior to the school year in which the

intervention occurs and prior to randomisation

l baseline and follow-up surveys should be undertaken with the action team, including items on

students’ social and educational characteristics.

Although this study found pen-and-paper questionnaires were feasible, were acceptable and produced a high response rate among students and staff, and some schools suggested that using school information technology facilities to undertake an online survey would not be feasible, we recommend that the cost of

new technologies (e.g. small, touch-screen‘tablet’ devices with 4G web capability) is reviewed prior to a

Phase III trial. Although there is an up-front cost in purchasing such equipment, such online survey methods may improve data quality significantly and deliver savings overall, as a result of minimising any costs associated with printing questionnaires and inputting, checking and archiving data. In addition to reviewing the economic costs and benefits, we would also consult the NCB YRG and teachers to get their views on this method, and on the risks and benefits of online surveys in schools, prior to making

a decision.

Quantitative data on intervention fidelity

Quantitative data on intervention fidelity should be collected in a Phase III trial. In addition to provider checklists and the documentary evidence that was collected in this pilot trial, the following data should also be collected:

l structured independent assessments of intervention delivery drawing on audio recordings and

observations of a sample of action team meetings, training sessions and curriculum sessions

l information on the professional background and other characteristics of each action group facilitator in

order to analyse how implementation and/or effects may vary

l measures of coverage regarding all relevant intervention inputs and outputs and, when relevant,

including baseline assessment. Qualitative data

Qualitative data should continue to be collected in a main trial as part of its integral process evaluation in order to assess unexpected processes, explore causal pathways and assess variation in implementation by context.

CONCLUSION AND RECOMMENDATIONS FOR FURTHER RESEARCH

NIHR Journals Library www.journalslibrary.nihr.ac.uk 84

Economic evaluation

Economic evaluation will be a core element of a Phase III cluster RCT. Our pilot economic analyses support the use of the CHU-9D scale with this population and the feasibility of CUA, although this should be supplemented with a CCA. However, at present, we do not believe that undertaking (complex) modelling to link observable trial outcomes to longer-term (health) events is warranted, given the inherent limitations with the existing evidence base. Such an exercise is likely to produce cost-effectiveness estimates that are so uncertain as to be of little practical use. Anonymised data linkage may also support longer-term

cost-effectiveness analyses via routinely collected health-service, education and criminal justice system data. The potential for collecting unique pupil numbers from schools at baseline, which could be linked to national health, education and crime databases, should be explored prior to a Phase III trial in order to facilitate long-term data linkage if possible. These findings suggest that student participation, particularly ‘having a say’ in revising the school rules, is a core component of the intervention, and therefore our economic analyses within a Phase III trial would also estimate these ongoing costs in order to facilitate potential longer-term analysis via data linkage.