Inference quality is the mixed methods term relating to quantitative research’s ‘internal validity’ and qualitative research’s ‘credibility’. Possible threats to the inference quality of this study include selection bias, differential attribution, experimenter bias, and the reliance on self-report. This section discusses the inference quality concerns for this study.
Selection bias is a possible threat to inference quality. Selection bias was not a threat for the RBNs as all RBNs in the sample agreed to participate. However, there was likely a selection bias in relation to participating women with breast cancer. It is possible RBNs did not invite some clients to participate, for example extremely distressed or elderly women. It may also be that women who had initially agreed to
19
The overall term reflecting the quality of mixed methods research is legitimation, which echoes quantitative research’s ‘validity’ term and qualitative research’s ‘trustworthiness’ term.
121 participate chose not to return their surveys when they experienced difficulties later. Other SCNS studies would have likely faced the same challenges.
In this study response rates amongst consenting women with breast cancer were relatively high (83% and 78% respectively), yet less than 30% of all possible –
although not necessarily eligible - women participated. That is, RBNs consulted with 239 women during the study period with 74 and 83 returning SCNS at each time points. The number of women turning down an invitation to participate is not known, nor is the number that nurses did not to invite. Additionally the number of women who may have been ineligible is not known (e.g. consulting with RBN after the specified data collection points).
This study was dependent on RBNs to recruit client participants. There are inevitably limitations of relying on healthcare staff for recruitment to research projects. Although RBNs expressed willingness to recruit client participants, this can be difficult in the context of a busy practice, as noted in other studies (Soothill et al., 2004). Thus, there were challenges in accessing client participants. For example, RBNs reported varied degrees of comfort and ‘success’ in gaining consent.
Differential attrition was another possible threat to inference quality in this study. Some clients did not follow through on their agreement to participate i.e. they did not return surveys. It could have been that woman who had initially agreed to participate did not complete surveys (and/or dropped out) as they were not coping well, whereas those who were coping well did return surveys. However, other SCNS studies would have experienced similar challenges.
Attrition may have also been an issue for RBNs. While not dropping out of the study, it is possible that completion rates of Study Patient Lists and Consultation Logs were less than complete. Although this was not the case with Day Logs (100% return rate), the researcher was unable to quantify the amount of missing
Consultation Logs and Study Patient Lists. As identified above, missing data undoubtedly existed (e.g. at least 13% of consultations went unrecorded in
122 Consultations Logs as opposed to Study Patient Lists), but the extent of missing Consultation Logs (and/or data points on Study Patient Lists) is unknown.
Experimenter bias is also a possible threat to inference quality in this study. The researcher’s perspective impacted what was studied, how it was studied, and the results of data analysis attained in the qualitative strand as well as the integration of quantitative and qualitative results to meta-inferences. The researcher’s
background and experiences outside of nursing, though in supportive cancer care systems, meant she may have had different insights than another researcher with another background. Furthermore, the researcher’s previous work with these interview informants could have had an impact on what, and the extent to which, RBNs chose to share. This is recognised as a potential bias. Yet, at most, only a minimal power relationship existed between researcher and researched at time of interview. As the researcher was familiar with the RBNs and generally with the health systems in which they functioned, it is likely that objectivity was not possible. On the other hand, insights might have been gained as the researcher had intimate knowledge of individuals and systems, and had discussed challenges with RBNs previously.
The research inferences benefited from the researcher’s knowledge of the nurse participants and contexts in which they worked. “A golden rule of making
inferences in human research is know thy participants! Having a solid understanding of the cultures of the participants and the research context is a valuable asset in the process of making inferences” (Teddlie & Tashakkori, 2009, p.289). These
understandings and knowledge assisted the researcher to interpret the results. Another component of inference quality that poses threats to this study is the reliance on self-report. The heavy reliance on accurate self-reporting is an
important limitation of this study. The SCNS relies solely on patients’ self-report, with no assurance of the accuracy of these reports. Similarly, Day Logs,
Consultation Logs, and Study Patient Lists rely on RBNs accurate recordings. It is possible that the data was inaccurate as all of the quantitative research tools relied
123 on self-report. Women with breast cancer may have positively framed their
responses in an effort to ensure RBNs received positive feedback, as women consulting BNs have generally been very supportive of them. It is also possible participating RBNs reports were skewed in order to positively frame their breast nursing practices.
With regards to SCNS self-reports, there are inherent challenges around individuals completing health need questionnaires. "Health needs is a deceptively tricky concept" which requires a "concise, valid and reliable tool" for evaluation (Asadi- Lari & Gray, 2005, p.294). Self-administered questionnaires are less expensive and intrusive than interviews, easily distributed and allow participants to complete them in their own time (Asadi-Lari & Gray, 2005). Yet self-administered
questionnaires can lead to misunderstandings and or superficial investigations with imprecise wording and limited interactions with participants (Asadi-Lari & Gray, 2005). Thus checks of validity and reliability are essential.
Although the SCNS tool was validated by its developers (Bonevski, et al., 2000), there were difficulties with this data collection tool identified by the researcher and RBNs in this study. For example, it was clear there was misunderstanding that surgery is considered a form of ‘treatment’ based on the inconsistent replies of women. Additionally, more than one nurse reported they believed women would misunderstand the scaling. Instead, RBNs suggested women would consider the numbers 1 – 5 reflecting intensity rather than truly considering the ‘satisfied’ component of the scale. Others have reported similar concerns with the SCNS scale20.
20
In June 2004, Ms Alison Zucca of the Centre for Health Research & Psycho-oncology reported respondents having problems with the complexity of scale while noting it had been 10 years since the SCNS’s validation during a presentation at the Behavioural Research in Cancer Control Conference.
124 Furthermore, it can be challenging to understand the meaning of SCNS results. The results can often only be understood in relation to with other results using the same instrument. Thus baseline or reference data is important21. Therefore,
comparisons have been made to other studies to add greater meaning to the SCNS results within this study.