Time-scale o f Study
One limitation of the design may have been that the study did not take place over a long enough time period to produce a size o f effect that the outcome measures could detect. Winzelberg et al. (2000) had found a significant effect over an eight week period and Barrera et al. (2002) over a 12 week period, both similar in length to the 10 week duration of this study. However, both of these studies were o f more structured support groups than that investigated in this study. It seems plausible that groups with less formal structure, such as that used in this study, may need an initial period in which to establish themselves, to develop their own structures and establish the roles taken by their various members. In support o f this contention, Lieberman and Russo (2002) found that naturally occurring on-line groups, similarly to face-to- face groups, had a core group membership that communicated the culture o f the group and provided leadership.
In their analysis of a naturally occurring group Salem and Bogat (2000) found that one o f the themes of the postings was discussion of the roles of group members. Although the data in the current study were not specifically analysed for this theme, the content analysis suggested that there was little discussion o f the group itself, and what there was, was mainly limited to expressions of appreciation o f the group or to particular group members. This could suggest that a group structure and culture was in the very initial stages of being established. If these aspects o f the group contribute to its hypothesised therapeutic effect, as is suggested by narrative approaches
(Rappaport, 1993) and by studies of face-to-face mutual help groups (Lieberman & Russo, 2002), then it could be that the group investigated in the current study was not yet functioning therapeutically at its highest level as it had not had time to establish its own structures or for roles or leadership to have been established amongst the group members.
Numbers o f Active Members Using the Mutual Support Group
A second factor that may have affected the functioning of the mutual support group was the relatively low number of active members. Forty per cent o f those in the Support Group condition visited the support group, of those who visited, two-thirds only read the messages and only a third (19 people) contributed to the board. Several of the contributors to the qualitative feedback about the group commented on how they would have liked more members and they hoped that the group would become more active as more people heard about it.
This pattern of participation, with a large proportion of the study participants using the group infrequently or not at all, was different from the participation patterns in the RCT’s conducted by Chang et al. (2001) and by Winzelberg et al. (2000) in which much larger proportions of participants in the experimental conditions used the support group. However, the mutual support groups in both these studies differed in significant ways from naturally occurring groups. It may be that the patterns of support group use found in this study are closer to those found in naturally occurring groups. This is in accordance with the findings of Lieberman and Russo (2002), who
report high overall turnover rates for those who participate in the on-line mutual support group they studied, but with a core membership providing leadership and giving continuity through their communication of a group culture.
So, while possibly being more representative of naturally occurring mutual support groups the fact that a relatively small proportion of those randomised to the Support Group condition were active users of the support group may have limited the study’s ability to detect differences in outcome.
Outcome Measures
A third factor could have been that inappropriate outcome measures were used. Humphreys and Rappaport (1994) stress the importance of using measures that will pick up changes that are perceived as important by group members, not just using measures designed for assessing clinical change in professional interventions. One of the measures used in the study the CORE, is designed for use within clinical practice, however the Satisfaction with Life Scale and the Sense of Community Index are both designed to assess non-clinical psychological factors, overall life satisfaction and sense of community inclusion, respectively. However, in defence of the measures used, the qualitative feedback and the quantitative assessments of satisfaction with the site did not produce results suggesting that participants’ perceptions of the help provided by the mutual support group differed greatly from those being picked up by the outcome measures used. Also, the relatively high overall level of participants’ scores on the CORE and the association between scores
on the Satisfaction with Life Scale and support group use suggest that these measures were picking up factors that were associated with looking for on-line psychological support and as such were appropriate to use in investigating outcome.
Another point, which may have affected the questionnaire data, is the method of its collection. All questionnaires were completed on-line. This was appropriate for the study but varies from the way that the questionnaires used for the reliability and normalisation data will have been collected. Although it has been shown that, in an undergraduate population, on-line completion of a brief questionnaire dealing with sensitive issues does not lead to significant variation from traditional pen and paper completion (Knapp & Kirk, 2003), it should be borne in mind that some unreliability with comparisons with the normative data may have been introduced by these varying methods of questionnaire completion.
It should also be noted that a relatively low proportion (47%) o f participants completed the post-measures. Although analysis of the pre-measures did not suggest that those who completed the post-measures differed from those who did not, it may be that some biases were introduced at this stage.