Qualitative data collection and analysis
The aim of the process evaluation was to examine the feasibility and acceptability of the NAP SACC
intervention and study design according to three prespecified progression criteria (see Chapter 2, Progression criteria). The design of the process evaluation used three relevant parts of the RE-AIM framework (Reach, Adoption and Implementation).99The key process evaluation questions (progression criteria) and the data
with key stakeholders (i.e. NAP SACC partners, nursery managers and staff, and parents), (2) non-participant observations (of nursery workshops and meetings between NAP SACC partners and nursery managers) and (3) document analysis of NAP SACC partner logs, Review and Reflect forms and goal-setting forms.
Non-participant observation
All observations were conducted by Jane Collingwood (trial co-ordinator, maternity cover), with the
exception of the NAP SACC UK partner training and one nutrition workshop (in nursery 3), both conducted by Rowan Brockman (research associate). The purpose of the nursery workshop observations was to assess the fidelity, feasibility and acceptability of the workshops by documenting its delivery and its receipt by nursery staff. Observations focused on who and how many staff attended, the workshop format, elements that worked well/less well, and the behaviour, interest and engagement demonstrated by the workshop facilitators and participants. Observations were recorded using a semistructured observation schedule in order to document prespecified information while also allowing for unstructured qualitative observations to be included.
The purpose of NAP SACC partner/nursery manager meeting observations was to assess the fidelity, feasibility and acceptability of these meetings and focused on the style of interaction (e.g. mutual decision- making or led by participant, supportive or critical, etc.), aspects that worked well/less well, and the behaviour, interest and enthusiasm displayed by the NAP SACC partner and manager. Observations were recorded using a semistructured observation schedule.
TABLE 6 Outline of process evaluation questions and methods
Progression criteria Methods Details
1. Was it feasible to implement the NAP SACC intervention with child-care providers?
Non-participant observations Semistructured interviews Document analysis
Observation of NAP SACC UK partner training (n = 1) Observations of nursery workshops (n = 10)
Observations of NAP SACC UK partner/nursery manager meetings (n = 5)
Interviews with NAP SACC UK partners (n = 4) Interviews with nursery managers (n = 12) Interviews with nursery staff (n = 4)
Analysis of completed logs (n = 5), Review and Reflect forms (n = 5) and goal-setting forms (n = 5) 2. Was the intervention acceptable to NAP SACC
Partners and the majority of child-care managers, staff and parents?
Non-participant observations Semistructured interviews Document analysis
Observations of nursery workshops (n = 10) Observations of NAP SACC UK partner/nursery manager meetings (n = 5)
Interviews with NAP SACC UK partners (n = 4) Interviews with nursery managers (n = 12) Interviews with nursery staff (n = 4)
Analysis of completed logs (n = 5), Review and Reflect forms (n = 5) and goal-setting forms (n = 5) 3. Were the trial design and methods
acceptable?
Semistructured interviews
Interviews with NAP SACC UK partners (n = 4) Interviews with nursery managers (n = 12) Interviews with nursery staff (n = 4)
Semistructured interviews
Semistructured interviews were conducted with key stakeholders (NAP SACC UK partners, nursery
managers and staff, and parents) to assess the fidelity, feasibility and acceptability of both the intervention and the research design. We aimed to interview all NAP SACC UK partners and all nursery managers, as well as one member of nursery staff from each intervention nursery. We also sought to interview 20 parents from both intervention and control nurseries.
Interviews with NAP SACC UK partners, nursery managers and nursery staff were conducted face to face or over the telephone, according to the participants’ preferences. All parent interviews were conducted over the telephone. Semistructured interview guides specific to each stakeholder group (Table 7) were used for all interviews to ensure that key topics were covered while allowing participants to discuss other issues they felt were important. Interviews were audio-recorded on an encryptable device and transcribed verbatim. All interviews were conducted by Rebecca Langford, with the exception of four parent interviews conducted by Ruth Kipping.
Document analysis
Three sets of documents were collected and analysed as part of the process evaluation: NAP SACC UK partner logs, Review and Reflect forms and goal-setting forms. NAP SACC UK partners were asked to log costs incurred in their participation in the NAP SACC UK study. Partners logged the date, type (e-mail, telephone, face-to-face meeting) and duration of contacts with each nursery. They also recorded travelling distance and time for the economic evaluation. These logs were analysed alongside other process
evaluation data in relation to the key progression criteria.
Review and Reflect forms were either completed individually by the nursery manager and then discussed with the NAP SACC UK partner or completed alongside discussion with the partner during one of their meetings. These forms were reviewed and updated at a follow-up meeting between the manager and partner. Data from these forms were used to document changes made within the nurseries and to triangulated with other forms of data (e.g. interviews).
Forms documenting the goals set by the nurseries were also collected and compared against data from the follow-up Review and Reflect forms and interview data to assess progress made.
TABLE 7 Details of semistructured interviews with key stakeholders Stakeholder group Number Method Key topic areas NAP SACC UK partners 4 In person (n = 3)
Telephone (n = 1)
l Views on nutrition, PA and oral health in nursery settings
l Experience of implementing the intervention
l Views on the future delivery of NAP SACC UK in future trial or roll-out
Nursery managers 12 In person (n = 9) Telephone (n = 3)
l Nursery background
l Views on nutrition, PA and oral health in nursery settings
l Experience of implementing the intervention
l Experience of participating in the research
Nursery staff 4 In person (n = 4) l Views on nutrition, PA and oral health in nursery settings
l Experience of implementing the intervention
l Experience of participating in the research
Parents 20 Telephone (n = 20) l Views on their child’s nutrition, PA and oral health at home and at nursery
Data analysis
Interview transcripts were entered into NVivo software to aid data management and analysis. Qualitative interview data were analysed using techniques associated with thematic context analysis and grounded theory. Initially, Rebecca Langford read through all transcripts to familiarise herself with the data. Rebecca Langford then coded four transcripts line by line to identify themes and develop an initial coding framework. Rebecca Langford and Ruth Kipping independently applied this coding framework to two further transcripts. The results from this independent coding were compared, any discrepancies discussed and revisions were made to the framework. Rebecca Langford applied the revised framework to all subsequent transcripts, making revisions where necessary in discussion with Ruth Kipping as appropriate. Observation data were entered into a Microsoft Excel®(Microsoft Corporation, Redmond, WA, USA) spreadsheet for analysis, with
these data being compared and contrasted with interview data, where relevant, to support cross-checking and triangulation. Data from NAP SACC UK partner logs, Review and Reflect forms and goal-setting forms were entered into the REDCap data management system. Data from interviews, observations and document analysis were triangulated to identify confirmatory or contradictory data.
All analysis was conducted by Rebecca Langford, in discussion with other members of the research team. Rebecca Langford is an experienced qualitative researcher who started work on the study to conduct the process evaluation after the intervention had been delivered and the majority of follow-up data had been collected. Rebecca Langford had no other role in the design or management of the trial or in collecting (quantitative) outcome data. This distance from the wider NAP SACC study allowed her to take a critical and objective view of the intervention and research process.