Análisis de las preguntas de la encuesta aplicada a los docentes

A strength of this thesis was the use of novel methods and strong research designs. Throughout, I have been guided by a framework that provided a comprehensive overview of the theoretical approaches that should be used to ensure adequate evaluation of patient information (Garner et al., 2012). Recommended methods that should be used in the tripartite model were lacking. I was therefore able to suggest a series of techniques that could be used to evaluate the gist leaflet within each stage of the framework. The relative success of these studies varied, however it is certain that methods suitable for lower literacy groups are needed to develop the framework further.

9.4.2.1 The think-aloud method

Few studies have used the think-aloud method in a cancer communication context. Although similar unstructured approaches have been used to assess the ‘cultural appropriateness’ of information materials, none have reported both qualitative and quantitative data (Cooperman et al., 2013). Despite this advantage, a number of limitations with the think-aloud method were noted. Participants who had lower levels of education found the task of reading aloud intimidating, particularly in the presence of a researcher. This is likely to have altered their reading ability, as well reduced the likelihood that they would report issues with comprehension. It also excluded participants who were not able to read lengthy documents, as the study protocol required people to read 13 pages of text within the interview. It is unlikely that this structured approach is how most participants would choose to read health information.

These limitations were apparent in hindsight and it is now clear that other methodologies may yield different results, particular when there is a focus on literacy barriers. These observations do not preclude the think-aloud method from being used in other contexts, but care should be taken not to exclude target audiences from providing their perspective. Modifications of the method could also

be made, such as removing the requirement to read aloud or providing more breaks during the interview. Further methodological requirement is clearly needed to improve the validity of the findings.

9.4.2.2 User-testing

As with the think-aloud technique, the user-testing methodology has not previously been used in a cancer communication context. A number of limitations were noted following its use. The nature of the method required participants to answer 8 true or false questions about CRC and CRC screening. This procedure was framed as a ‘test of the information material’ and not of the individual being intervie ed. Despite this, participants who were unfamiliar with tests and examinations may have felt under pressure to answer the items correctly. Participants were reassured throughout the study, but this pressure may have reduced how forthcoming they were when asked to disclose problems with the gist leaflet.

The use of true or false questions and small sample sizes meant that chance variation may have led to the gist leaflet being prematurely accepted as ‘fit-for- purpose’. Although this limitation as lessened by the progressively greater representation of participants with a low level of education in rounds 2 and 3 of the study, it should be recognised as a problem of the method. One alternative could be to adapt the method and use open-ended questions with standardised marking criteria. In addition to making the task more difficult, it may also provide an insight into the thought processes that occur when answering the question.

It should be noted that participants were not exposed to ‘The Facts’ booklet during the user-testing study. This was a conscious decision, as I felt that data collection would be maximised if participants were only able to comment on one information leaflet. Ho ever, providing ‘The Facts’ booklet as ell ould have provided a more ecologically valid test of the gist leaflet. It would also have provided insight into how the public would cope with being provided two information materials instead of one. Again, this decision may have resulted in a premature acceptance of the gist leaflet’s design and content. Future research should be a are of the context in which the intervention will be delivered throughout the design and evaluation stages.

9.4.2.3 Randomised controlled trials

A further strength of the thesis was that two studies used multicentre randomised designs. The randomised designs added validity to the conclusions of the studies and are considered the gold standard approach for testing public health interventions (Sackett et al., 1996). The use of multiple recruitment centres in all studies (but specifically studies 3 and 4) further increased the validity of the findings by reducing the likelihood that study effects would be artificially inflated (Bafeta et al., 2012; Dechartres et al., 2011).

It is often not feasible to collect socio-cognitive data within organised screening programmes. Study 3 was therefore used to provide complementary data for the national trial. However, this study was limited by the lack of follow-up and the use of screening intention as a proxy for screening participation. Despite the purposive sampling method used, study 3 was performed on a relatively educated and motivated sample. Combined, these limitations prevented me from examining the true effect of the gist leaflet on screening outcomes.

Although the data recorded in study 3 did not provide evidence for the communicative effectiveness of the gist leaflet, the national trial was still undertaken (Study 4). The size of the trial provided sufficient power to test novel hypotheses such as the extent to which the gist leaflet addressed SES inequalities in screening uptake. Similar studies investigating the effect of an intervention among low SES groups have shown encouraging trends, but were hampered by a lack of statistical power (Wardle et al., 2003).

Study 4 used a cluster randomised design as it was not feasible to run a parallel trial within the NHS BCSP. When the initial programme was funded, a parallel design was proposed to the National Institute of Health Research (NIHR). However, Real Digital International and the hubs requested that this was changed due to logistical challenges. Cluster designs have their own limitations including limited statistical efficiency that can be dealt with by using large samples and increasing the number of clusters (Campbell et al., 2007; Donner & Klar, 2004). This was achieved within study 4, thereby limiting the importance of this limitation. A more important point to note is the number of concurrent initiatives that were likely to have occurred during the study period. Cluster randomised trials are more susceptible to bias because of

differences between clusters (in this case days). Furthermore, these differences are difficult, if not impossible, to control for.

This highlights the important compromise that had to be made when developing this trial. On one side there was the argument that a trial should be designed optimally so that it has the best chance of demonstrating its true effect (i.e. a parallel design). On the other, there was the practical argument that the resources required to design the optimal trial would be so large that it may not be practicable. It also limits the extent to which interventions in the future could be evaluated within the national programme. In the end, we have to acknowledge that using a cluster design was a limitation of the trial and this should be considered when interpreting the findings.

Despite these limitations, study 4 was an interesting learning experience for all of those who were involved. It provided evidence that the collaborative efforts of hub staff and academics from a range of difference disciplines can come together to test interventions within the NHS BCSP. I hope that the hubs were also able to learn from the experience, and can accommodate future interventions within the screening programme.