The positivist biomedical paradigm provides a theoretical foundation for health questionnaires. This can be readily observed in the common aim to gather objective data on variables of interest, then, using statistical methods, to correlate with clinical diagnoses or health outcomes. However, it could also be argued that questionnaires go beyond the narrow limits of the biomedical model in the way they give credence to the subjective experiences of research participants. For example, a number of questionnaires have been developed to measure highly subjective phenomena such as pain and anxiety.
Foddy129 notes that a stimulus-response model provides a conceptual framework which underpins the collection of ‘objective’ data in questionnaire research. This model refers to the theory that research participants will respond in comparable ways to a standardised stimulus (usually a single questionnaire item). Under this model, the researcher makes a number of implicit assumptions. Firstly, it is assumed that questions are understood in the same way by all respondents. It is also assumed that respondents will interpret questions in the way that the researcher intended them to be understood and that research participants will respond in the same way. Questionnaires typically utilise closed questions in order to fulfil this requirement for standardised responses. The standardised responses are then assumed to represent facts in the real world, rather than written interaction between the researcher and respondent. Perhaps the most important understanding required under the stimulus-response model is the assumption that standardised responses given by different respondents can be meaningfully compared with one another. All quantitative questionnaires and surveys operate under these assumptions, whether or not this is actually acknowledged by the researchers.
Questionnaires are typically informed by at least one other theoretical perspective in addition to the stimulus-response model. For example, Brooks168 discusses the ways in which Lawton’s (1972) behavioural model informed development of Katz’s (1983) Instrumental Activities of Daily Living measure (IADL). Similarly, theories of well-being have informed the development of health-related quality of life measures (HRQoL). Fayers & Machin169 describe how Calman’s (1984) expectations model, which takes into account an individual’s aims and goals in life, provided the conceptual basis for inclusion of personal values in the Patient Generated Index (PGI).
Symptoms questionnaires are usually informed by the stimulus-response model and the ‘common sense symptom perception model’ (described in Chapter One), both of which are grounded within the positivist biomedical paradigm. Under the common sense symptom perception model it is hypothesised that women with ovarian cancer are able to perceive symptoms of the disease. Furthermore, the severity, frequency and number of symptoms perceived by women should increase with advancing stage of ovarian cancer.
4.2.1 EORTC questionnaire development methodology
A number of questionnaire development textbooks and articles were consulted throughout the research period.129-130 168-176 These outlined rigorous questionnaire development methods and described essential components of the validation process. Reading this literature identified the European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Group questionnaire development methodology as well established in producing valid and reliable tools for measuring cancer-related symptoms.169-170 Furthermore, EORTC questionnaire development methodology fulfils six of the seven criteria described by Guyatt et al. (1986, in Brooks168) as a ‘Rolls-Royce’ model of questionnaire development. The one criterion not met is the requirement for questionnaires to contain Likert scales consisting of seven to 10 points. This number of categories is also described by Streiner & Norman130 as ideal for good reliability of questionnaires. By contrast, EORTC questionnaire methods specify, ‘items should preferably employ four-point Likert scales’.177 Moreover, validation of four-point
Likert scales by the EORTC group among diverse cancer populations has proven good reliability.178-183
The EORTC approach to questionnaire development was also adopted on the basis of demonstrated success in producing a valid and reliable measure for assessing symptoms in ovarian cancer patients.184-185 The EORTC ovarian cancer module (QLQ-OV28) was developed to assess treatment-related symptoms and quality of life. Unfortunately, the post-diagnosis focus of the questionnaire meant that the tool itself was inappropriate for our own research.
On the basis of this information, it was decided to use an EORTC questionnaire development methodology. Guidelines published by the EORTC Quality of Life Group informed the research described in this chapter.177 186-187 However, some aspects of the methodology were not appropriate as the ovarian cancer symptoms questionnaire was not designed as a psychometric measure. The next section describes the principal components of questionnaire validation.
4.2.2 Questionnaire validation methods
Questionnaire development methods describe a number of different components of validation. Face, content and construct validation are requisite to demonstrate the acceptability and usefulness of any health questionnaire, while reliability analyses are essential to establish the consistency of a tool.
Face validation is the fundamental component of questionnaire validation. This assesses whether a questionnaire is readable, appears to be sensible and whether the items in a tool appear to measure the construct they claim to measure. Questionnaires should be visually appealing, with an uncluttered design. Clear instructions showing respondents how to complete items should always be included and the basic format should be readily understandable. Face validity is vitally important as it promotes acceptance of the questionnaire among research participants. It is only after a questionnaire has demonstrated face validity that other aspects of validity can be investigated.
Content validation assesses whether the items in a questionnaire are relevant and whether it is comprehensive enough to adequately investigate the subject it is designed to measure. Convergent validity, also referred to as construct or trait validity, refers to the consistency with which individual items in a questionnaire measure the same theoretical construct. While there is no single experiment to prove construct validity, a number of different internal consistency analyses may be undertaken to assess convergent validity.
Questionnaires which set out to measure a single construct, such as depression, should have high internal consistency. This is determined by assessing correlations between items in a questionnaire, and the extent to which they correlate with an overall score. Cronbach’s alpha statistic, sometimes using a split-half method, is often utilised for this purpose. Factor analysis typically follows correlation analyses in the development of psychometric scales. This allows simplification of a large number of correlated variables based on underlying constructs or ‘factors’ in order to reduce the number of items in a scale.
Concurrent validity, also referred to criterion or discriminant validity, involves assessment of a questionnaire against an established ‘gold standard’ to determine whether the new questionnaire measures the same construct. In clinical trials, discriminant validity refers to the ability of a health measure to distinguish clinically significant outcomes in therapeutic responses among cases and controls.168 For example, the EORTC core questionnaire (QLQ-C30) has been used to assess treatment-related quality of life outcomes in randomised controlled trials conducted in 35 countries.188
Test-retest reliability refers to the ability of a questionnaire to consistently capture equivalent information over time, other things being equal. Depending on the intended use of the questionnaire, reliability is determined either by different observers using the same tool to assess the same person (inter-observer reliability), or by a single observer using the questionnaire on different occasions (intra-observer reliability). For questionnaires completed by research participants, rather than the researcher, intra-observer reliability is assessed. Respondents are
asked to complete the same questionnaire within a time-frame when the outcome of interest is expected to be stable. The test-retest interval should not be too short or the respondent’s memory will influence responses but it should also not be too long, or the phenomenon being measured may have changed. The intraclass correlation coefficient is commonly used to assess the reliability of health questionnaires. High reliability coefficients indicate that a measure has good stability. Questionnaires with low reliability should either be redesigned and re- validated or excluded from use.