There are several possible explanations for the modest, statistically non-significant effect of the intervention, which can be conveniently grouped as study-specific factors and the nature of the target group. Before exploring these explanations, the results of the present study need to be put into the context of existing and recent research findings, particularly the recent studies on alcohol brief interventions and text message studies to reduce alcohol consumption.
Recent studies on alcohol brief interventions
Evidence from two systematic reviews5,6shows that alcohol brief interventions are effective, although the
effects are small. The reviews found that the proportion of men who engaged in binge drinking (heavy
drinking) was reduced by 11%5and 12%.6In contrast, this study found that the net reduction in binge
drinking was 5.7%. It also found that the reduction in binge drinking in men recruited through general
practice registers was higher (8.6%, 95% CI–18.7% to 1.5%) than in those recruited by TSS (2.1%,
95% CI–13.5% to 9.4%). Both of the reviews5,6concluded that there was no evidence that brief
interventions are effective with dependent drinkers.
Since the publication of these reviews, five large trials7–10,188of alcohol brief interventions, which found
that the intervention was ineffective or possibly harmful, have been published. Three of these trials formed the SIPS (Screening and Intervention Programme for Sensible Drinking) research programme in which the
intervention was evaluated in primary care,7accident and emergency departments8and criminal justice.9
All three studies found that the brief intervention was not effective, as the primary outcome was similar
in the active treatment and control groups. A separate study10in primary care found that, at follow-up,
the intervention group had a statistically significantly higher alcohol consumption than the control group.
Finally, a complex study188that addressed four risk behaviours (smoking, alcohol use, exercise and healthy
eating) found no significant effect on alcohol use. The last of these did report that the intervention increased intentions to change, the number of attempts at change and perceived success in achieving change, but these did not result in measured behaviour change.
Text message studies to reduce alcohol consumption
Several recent studies have explored the use of text messages to tackle hazardous or harmful drinking.
Two studies, one with vocational school students81and the other with mandated college students,189
used an uncontrolled before-and-after design, making it difficult to draw conclusions about effectiveness.
Four studies78,190–192on text messaging were conducted with college students and none showed a significant
effect on alcohol consumption. One study190stated that there was no significant effect of the intervention;
another study,192which intended only to assess feasibility and acceptability, also reported no significant
effect on alcohol. The third study191reported a significant effect in a subgroup, and, finally, a very small
study78reported a significant effect on a secondary outcome measure (readiness to change drinking).
A further, large study193of university students, which compared two smartphone apps with a control
group, found that one of the apps significantly increased alcohol consumption.
Text message studies have also been conducted outside the educational environment. One study76of
emergency department attendees claimed that a text message intervention can lead to small reductions in the self-reported number of heavy drinking days and in the number of drinks consumed per drinking day. However, the authors reported that there was differential loss of heavy drinkers at follow-up (more heavy drinkers were lost in the intervention group). After adjustment for the losses by multiple imputation,
the intervention effect became non-significant. Finally, a pilot study82on dependent drinkers found a
statistically non-significant difference in favour of the intervention. In summary, although the technology appears to work well, there is a lack of convincing evidence that text messages are effective in reducing alcohol consumption.
Potential limitations of the study
Randomised controlled trials can suffer from problems of design and conduct, which impair their ability to detect the effect of interventions. The following section places particular attention on the biases identified
in the Cochrane risk-of-bias tool194and those for which there is strong evidence of bias.195It also follows
the recommendation of the Cochrane group and considers those design features of a trial that could have contributed to the inconclusive findings.
EXPLANATIONS FOR THE MAJOR FINDINGS AT FOLLOW-UP
NIHR Journals Library www.journalslibrary.nihr.ac.uk 68
Insufficient statistical power
This study was large, much larger than most published trials on alcohol brief interventions.5,6The sample
size calculation assumed that 20% of men would be lost to follow-up, but in the event only 14% did not complete the final follow-up. Thus, the study had a higher power than was stated in the original protocol. However, the power to detect an effect could have been reduced by the large fall in alcohol consumption seen in the control group (and, by implication, the intervention group), which could increase the variance of the outcome measure.
Lack of blinding of participants to the purpose of the study
If study participants are aware that the purpose of a study is to reduce alcohol consumption, they may under-report their consumption to please the researcher. In this study, participants were told that the research was investigating alcohol and health and that the aim was to identify ways to reduce alcohol consumption. Thus, demand characteristics, or social desirability (described above), could have encouraged them to report a lower alcohol consumption than was the case. This could result in non-differential misclassification of the outcome with the most likely effect of biasing the estimate of treatment effect to
the null.196,197It is unclear whether this form of bias to the null could explain the study findings. All trials of
alcohol brief interventions would have suffered from the same bias, but many detected significant effects of treatment.5,6
Poor allocation concealment
The nature of the treatment group should be concealed from study participants and from the observers making the baseline and follow-up measurements. In drug trials, blinding of participants can cover the look, taste and smell of the medication. In this study, an attempt was made to conceal treatment status (intervention or control) by providing the two groups with a similar number of text messages. However, the content of the text messages received by the intervention group would have revealed that they were being encouraged to reduce their drinking. This might have resulted in a greatly increased social desirability effect in the intervention group, leading to a greater reduction in their consumption. This was not the case.
Another possibility is that awareness of the purpose of the messages induced psychological reactance,198
reducing social desirability or even encouraging participants to exaggerate their consumption. Neither possibility seems likely because of the similar high level of satisfaction with the study methods found in both groups. Furthermore, the effect would need to be sustained to the 12-month follow-up. This seems
unlikely, as the effects of most behavioural interventions attenuate over time.113
A further feature of allocation concealment is that the researchers making the baseline and outcome
measurements should be blind to the participants’ treatment status. Randomisation in this study was by a
remote web-based system to which none of the researchers had access, and so they remained unaware of treatment allocation while the outcomes were being measured.
Data collection methods
One limitation of this study is that it used self-reported alcohol consumption, which could be influenced
by social desirability bias.168The validity of self-reports of alcohol consumption is a longstanding concern of
researchers. There was a consensus that self-report methods provide a reliable and valid way of measuring
alcohol consumption.199However, more recent studies have drawn attention to the importance of
measuring the strength and volume of drinks132,133and episodes of binge drinking.200Thus, we modified
the data collection method, timeline follow back,130to collect detailed information in order to increase the
accuracy of the estimates of alcohol consumption.
A change in data collection methods between baseline and follow-up might have influenced the results. Efforts were made to minimise this through staff training and ensuring regular contact between research staff to ensure a consistent approach. The data on alcohol consumption were collected using the same questions at the baseline and follow-up interviews.
Loss to follow-up
Alcohol brief intervention studies commonly have average loss to follow-up rates of 27%, with a
significantly higher loss to follow-up in the intervention group.5It is suggested that a loss to follow-up of
> 20% gives rise to concern.201,202In this study, loss to follow-up was 14% and almost identical in the
intervention and control groups. Furthermore, the finding that, among those lost to follow-up, alcohol consumption was similar in those in the intervention and control groups suggests that bias due to loss to follow-up is unlikely. Finally, the multiple imputation to take account of missing data produced an almost identical treatment effect to that from the main analysis. Thus, loss to follow-up bias cannot explain the study findings.
Poorly designed intervention
The intervention had a strong theoretical and empirical basis. The intervention was designed around the
HAPA,87a behaviour change model that describes the adoption, initiation and maintenance of a new
behaviour as a process that involves a motivational and a volitional phase. It also incorporated behaviour
change techniques from the taxonomy developed by Michie et al.92The intervention drew heavily on the
components of alcohol brief interventions, which have been shown to have a modest but significant effect
on mean consumption and binge drinking.5,6
A possible weakness of the intervention is that it may have been too subtle. It used the technique from
motivational interviewing, raising the participants’ awareness of the discrepancy between the impact
of their current drinking and their social roles and responsibilities. Possibly a more direct approach, emphasising the harms of drinking and accompanied by advice to cut down, would have been more effective. However, it is also possible that such an approach could cause some irritation to participants, as text messages from someone they do not know telling them that they drink too much might not be welcomed. Such an approach is commonly used in health-care settings, where a doctor may give such advice, but could be less acceptable in a community-based study.
Inappropriate delivery method
As alcohol brief interventions delivered face to face are effective, it is possible that the weakness lies in the use of text messages for delivery. A cardinal feature of text message interventions is that they are delivered in small packages over an extended period. In contrast, conventional alcohol brief interventions are
commonly delivered in a single session or a few linked sessions. Possibly the separation of the components of text message intervention over a 12-week period reduced the overall effectiveness. If, for example, texts increasing risk awareness were distant from those encouraging goal-setting and action-planning, then the motivation for action would be reduced. However, in this study that was not the case. Texts aimed at motivation were spread throughout the intervention: indeed, there was regular reinforcement of key steps in the behaviour change sequence. Furthermore, behaviour change interventions delivered by mobile phone have been shown to be effective in some circumstances, particularly smoking cessation and
adherence to treatment in patients with human immunodeficiency virus infection.66
Poor fidelity of delivery of the intervention
A major advantage of text message interventions is that they are sent automatically to a participant’s
phone. Electronic monitoring of this revealed that almost all of the text messages had been received by the
participants’ phones. Furthermore, the frequency of responses to the messages confirmed that participants
had opened and read the texts.
Poor engagement with the intervention
The study was designed to encourage engagement of the men with the intervention. Several approaches were adopted: using language tailored to the target group, sending welcoming text messages and
using participants’ first names in text messages. The intervention was also designed to be interactive by
prompting men to send responses to some of the text messages they received. These responses showed high levels of engagement with key components of the intervention.
EXPLANATIONS FOR THE MAJOR FINDINGS AT FOLLOW-UP
NIHR Journals Library www.journalslibrary.nihr.ac.uk 70
Limited duration of intervention delivery
The intervention was delivered over a period of 3 months. Possibly an intervention delivered over a longer period would be more effective. Disadvantaged individuals are less likely to translate intentions to change
into action to modify behaviour.30Thus, a longer-term intervention could place more emphasis on the
volitional components of behaviour change and reinforce commitment to reducing alcohol consumption.
Attentional control
The study used an active or attentional control which could have influenced drinking behaviour. The control group received almost as many texts as the intervention group and the texts were focused on health, which may have made men think about their health and their drinking. The control group were as likely to engage with their text messages. However the questions in their texts were mainly of a type found in trivia quizzes and had a multiple choice format. Responses to these questions only required a single letter answer. In contrast the questions in the intervention were more challenging, which required review and disclosure of drinking behaviour. Participants gave considered responses to the questions which indicated engagement with the key steps of the behaviour change process.
Research participation effects
McCambridge et al.177have suggested a novel mechanism by which research participation could bias the
estimate of treatment effect. Suppose only a small number of individuals are susceptible to behaviour change, and that most of them respond to the research participation effects by reducing their drinking. This will occur to an equal extent in the intervention and control groups, but it will leave a reduced number of participants who would respond to the effects of the intervention, biasing the observed treatment effect to the null. At present this is just a hypothesis, but if true it suggests that brief alcohol intervention studies underestimate the real effect of the intervention. Under this hypothesis, the null effect in the present study could have occurred if research participation effects were larger than in previous studies, or if the intervention were much weaker than in other studies. Again, however, the effects of this mechanism might be expected to attenuate over time, and it is unlikely to be able to explain the sustained fall.
The nature of the target group
Behaviour change interventions are less effective, or ineffective, in disadvantaged/low-income groups,
although they can still have modest statistically significant effects.20–22Evaluations of smoking cessation
interventions suggest that the lower effectiveness is due not to lower initial uptake, but to lower sustained
compliance with the intervention.28,29Fear of being judged and fear of failure have also been identified as
barriers for disadvantaged groups.32
Possibly the social and physical environment in which the men lived could have overwhelmed their
intentions to drink more moderately. There is convincing evidence that area characteristics influence health
outcomes.203–205A review of interventions to promote smoking cessation found that there were more
barriers to change for disadvantaged groups, particularly pro-smoking norms, additional cues to smoking
and increased stress.23Qualitative research suggests that disadvantaged individuals who live in poorly
resourced and stressful environments are isolated from wider social norms and have limited opportunities for respite and recreation.31
Living in a disadvantaged neighbourhood could also lead to heavier alcohol consumption.206Poverty and
poor neighbourhoods can increase stress,207which in turn increases the frequency of binge drinking.208
Low-income groups encounter a higher frequency of daily hassles209and alcohol outlets are more common
in disadvantaged neighbourhoods.210Stress and alcohol cues (such as the ready availability of alcohol)
could increase motivation to drink.211More research is needed into the impact of these barriers on
intervention effectiveness, and on strategies to overcome their effects.34
An interesting finding from a prespecified analysis was that the intervention had a much larger effect in men recruited from general practice registers than in those recruited through TSS. The latter group were
more likely to be unemployed and single and to have lower educational attainment. They were also more likely to be heavier drinkers. Thus, the finding of lower effectiveness in the TSS group is consistent with the explanation that stress and the hassles of daily living reduced the effectiveness of the intervention. If true, it raises the question of whether it would be better to intervene on men recruited from general practice registers, because effectiveness is higher, or whether the TSS group, whose need may be greater, should receive higher priority. As all of the men in the study are at high risk of alcohol-related harm, this a challenging ethical issue.