Key cost domains
To enable an economic evaluation to be conducted as part of any future effectiveness trial, we sought to map, in as much detail as possible, the key cost domains.
The costs associated with running KAT can be mapped on to three levels, namely families, schools, and external organisations. We deal with each in turn. For families, the main costs incurred relate to the opportunity cost of attending the family event and the expenses incurred through travelling to and from school. Some parents/carers might also incur childcare costs for young siblings of children involved in KAT, though the family events are open to all family members.
Schools’ costs fall into three clear categories. First, there is the cost of staff time devoted to the
programme. This comprises teaching staff, teaching assistants, and non-teaching staff such as caretakers. Second, there is a cost attached to the production of materials for the family event. These are mainly paper
and stationery– there is no requirement for schools to buy equipment as part of their classroom
preparation. Third, by delivering KAT, schools incur an opportunity cost– i.e. the time spent on the
programme by staff and pupils could have been utilised for other curricular or non-curricular activities. During this exploratory trial we were also able to map out which external organisations were involved in programme delivery, the nature and extent of their input, and how this might evolve as part of the systems and structures established for any future effectiveness trial. In the current study, the following agencies supported the implementation of KAT:
l A Healthy Schools Scheme officer, who assisted with recruitment of schools, and also attended some
of the training events for school staff and the family events.
l An educational consultant who provided training and support for school staff and also assisted with
the running of the family events. The cost of her employment was covered by a grant from the Welsh Government.
l A local drugs agency which provided an information stand at all of the KAT events.
l A local secondary school which loaned its‘smoothie bike’ free of charge for use at the family event in
all of the intervention schools.
A number of non-staff costs were also covered by external agencies (principally, by a grant from the Welsh Government). These were the updating and printing of the programme manual and the purchase of additional educational resources; the cost of providing refreshments at all three schools which ran family events; the purchase of additional display boards (which were used in multiple schools); and the cost of
van hire to transport the‘smoothie bike’ between schools. The ‘goody bags’ and their contents (including
the DVD) had already been produced and paid for by Gwent Police before the start of the trial as part of the earlier phase of programme delivery; Gwent Police made these items freely available.
Questionnaires completed by pupils in this study included questions on whether or not they completed the KAT classroom work, and rates of attendance at the family event (for pupils, families, and the number of adults present from each family). This information on programme reach could be used alongside data on programme costs to ascertain the cost per participant/family.
Translating inputs into financial costs
Based on our experience in this exploratory trial, we believe that it should be possible to translate all of the key inputs into financial costs which could then be used as part of an economic evaluation. A section of the programme handbook could be used to ask staff to record (in a pro forma in the book) how many hours they spend on KAT (pupil contact time and other preparation time). It should be possible for a process evaluation within any future effectiveness trial to record total pupil contact time in a subsample of the intervention schools, so that teacher self-reports and researcher observations can be compared.
Although we did not ask schools to calculate the cost of teacher input into the programme, we believe that it would be feasible to ask head teachers either to provide this information or to share sufficient information for us calculate an hourly cost.
In this exploratory trial, the cost of refreshments, transportation of the‘smoothie bike’, and other non-staff
costs were paid directly by the research team from a centrally held programme grant. In any future
effectiveness trial, we would expect schools to organise and pay for refreshments at their family event, and
to submit invoices for these and other costs (such as those associated with the use of‘smoothie bikes’) to
a central programme co-ordinator. These data could, therefore, be used to provide accurate information on the non-staff costs which schools need to recoup.
Process evaluation observation of classroom preparation activities allowed us to generate an overall impression of the amount of consumable materials (e.g. paper, pens and posters) that schools used. Given the difficulty of producing exact costs for these materials, the most efficient way of estimating the costs would be to ask each school to provide an estimate of all of the non-staff expenditure items they provide. This information could be requested at the same time as schools submit invoices for reimbursement of costs paid by a central programme grant. It was feasible to separate research-related and programme delivery-related costs in this exploratory trial. A health economic evaluation as part of any effectiveness trial would also need to ensure that all costs could be allocated to the research process or the delivery of the KAT programme.
Suitability of primary and secondary outcomes for an economic evaluation
In a future effectiveness trial it should be possible to analyse primary outcomes (relating to alcohol consumption) and secondary/tertiary outcomes (relating to family communication) alongside data on costs relating to programme delivery. We would be likely to adopt a similar approach to that currently being
used in an ongoing trial of the SFP10–14, in which a cost–consequences analysis is being used. However,
given the problems experienced with recruitment of parents into the current trial, it is unlikely that we would be able to access data from parents on service utilisation, which is a component of the data
collection from parents in the SFP10–14 trial. Traumatic, rather than medical, problems are common
consequences of adolescent alcohol misuse, and so estimating service use would focus on accident and emergency admissions 2 years post intervention (or after other specified follow-up intervals).
Feasibility of the data collection instruments
Our experience in this exploratory trial suggests that it is feasible to collect comprehensive and reliable data concerning programme delivery costs, and intervention reach and engagement. Low rates of missing data for the questionnaire items in this study which would form the primary and secondary outcomes in any future effectiveness trial suggest that it would be feasible to link these data with programme cost data. However, it is unlikely that we could collect service utilisation data from large numbers of parents using the methods employed in this exploratory trial, given the low recruitment rates encountered.
Chapter 4 Discussion
P
arts of this text are reproduced with permission from Segrott et al.1Introduction
In this chapter, we discuss our key findings. We begin by outlining the study’s main limitations. The chapter
then discusses the study’s findings, structured around our progression criteria – starting with those relating
to programme implementation, followed by the criteria concerned with a future effectiveness trial of KAT. For each criterion we provide a summary of the evidence generated by the study, and also map this onto
the study’s objectives. This is followed by an overall assessment of our programme- and trial-related
progression criteria. We then situate our findings in relation to the existing literature on school-based alcohol misuse prevention programmes.