This was initially precipitated by extensive - technological developments that expanded all aspects of disease management, giving rise to an increase in the variations of clinical practice and escalating costs. As a direct consequence, there has been an ever-increasing demand on clini- cians to demonstrate the benefits of their inter- vention. More recently, the need for evaluation has regained renewed momentum as an empha- sis has been placed on actively involving patients in making decisions about their own health care provision (Hatch 1997). This process requires information to be available to an even wider group of people to enable informed decision making about the relative benefits of different health care interventions.
A framework for measurement
The World Health Organization's model of ill- ness provides a useful framework and a common terminology for describing and measuring the consequences of diseases and the impact that rehabilitation interventions may have on them (WHO 1980). The original framework, termed
The International Classification of Impairments, Disabilities and Handicaps (ICIDH), is now widely
used as a basis for evaluation within both clinical and research practice. Recently it has been exten- sively reviewed in order to avoid a purely medical interpretation of disablements and to take into account the role of the environment in the disablement process (Ustam & Leonardi 1998). The new terminology of The International
Classification of Impairments, Activities and Participations (ICIDH-2, WHO 1997) is:
• Impairment describes a loss or abnormality of
body structure or of a psychological or physi- ological function.
• Activities describe the nature and extent of
actual performance in functional activities, at the level of the person.
• Participations describe the nature and extent of
a person's involvement in life situations in the actual context in which they live, at a societal level. It recognizes the complex relationship between a person's health condition, their impairments and activity restrictions.
Figure 2.2 illustrates how this framework may be applied to physiotherapy practice.
General principles of outcome
measurement
The validity of outcome measurement is depend- ent upon the rigour of the measurement process and the validity of its measures (Rudick et al 1996). It is therefore necessary when undertaking an evaluation to consider a number of important factors. These are discussed in detail in the fol- lowing sections.
Determining the purpose of the evaluation
When initially deciding what to measure it is important to be clear about the purpose of the
evaluation and the information you wish to gain. This might be for a number of reasons. You might wish, for example, to routinely gather baseline and discharge data on all patients attending the service, regardless of their dis- order, for the purpose of identifying key infor- mation about the individual, and for service evaluation and audit. Generic disability assess- ment scales such as the Barthel Index (Mahoney & Barthel 1965) or the Functional Independence Measure (Granger et al 1993) are commonly used for this purpose. On the other hand, you may wish to evaluate the impact of a specific intervention on an individual's specific prob- lems. In this case the choice of measure (or combination of measures) may be quite differ- ent. For example, if the purpose was to evaluate the effectiveness of a splinting programme in a patient who presented with loss of joint range due to soft tissue shortening, the most relevant measure might be the measurement of a joint angle using a goniometer (Norkin & White 1975). It is likely that this would be combined with a measure of function relevant to the patient's particular problems, such as the ability
to stand up or to walk. These two examples illustrate how the outcomes chosen may differ according to the purpose of the evaluation.
Although a combination of outcomes is often necessary to comprehensively evaluate your intervention, it is important to be selective, remembering that neither yourself nor the patient should be overburdened by excessive measurement. Research studies are better able, in most circumstances, to investigate the effective- ness of interventions in more detail and, where necessary, with more sophisticated equipment.
Selecting relevant outcomes to measure
In many neurological conditions long-term dis- ability is the norm. For this reason outcome mea- surement can be broadly viewed in two ways: first, the extent to which stated aims of inter- vention are achieved; and, secondly, the extent to which adverse events might occur if the treat- ment is not given. The range of possible out- comes is presented schematically in Figure 2.3. Death is an easy outcome to measure, but this rarely has direct relevance to physiotherapy
Figure 2.3 Range of possible outcomes following brain
trauma (reproduced from Pope by kind permission).
management. More difficult to measure is the success or failure of the longer-term con- sequences of the disease with regard to mor- bidity. Pope (1992) refers to these in terms of 'dynamic success' (for instance the ability to walk again, to regain function, to get back to work); 'static success' (where the patient's condition is well maintained and where secondary complications do not occur); and 'failure' (for instance the development of pressure sores, con- tractures or pain). Bax et al (1988) illustrated the importance of static success in their study exam- ining the health problems of a population of physically disabled young people in the com- munity. Of the 104 subjects studied the secondary complications experienced included contractures of lower joints (59%), deformed feet (25%), urinary incontinence (59%) and pressure sores (33%). Many of these complications are avoid- able and, hence, a successful outcome in terms of static success would have been attained if they had not occurred.
In the majority of cases, however, the aims of therapy are not simply to achieve static success but also to improve function and well-being. For this purpose it is important that the measure- ment of outcome is focussed at the level at which the intervention is intended to effect change. For example, interventions aimed to impact on impairments such as muscle weakness should measure strength; interventions aimed predomi- nantly at improving function should measure function. To be successful, this requires the ther-
apist to have a thorough understanding of the intervention, and to be specific in identifying in advance what the intervention aims to achieve.
Selecting appropriate measures
In selecting which measure to use, a number of factors must be considered. It is not the purpose of this chapter to describe these in detail, but rather to provide an outline of the more important issues. Several textbooks (McDowell & Newell 1996, Wilkin et al 1992, Streiner & Norman 1995) and articles (Medical Outcomes Trust 1995, McDowell & Jenkinson 1996, Fitzpatrick et al 1998) address these issues comprehensively in a more detailed and technical manner.
In brief, in choosing an outcome measure, four key factors should be reviewed.
1. The purpose of the measure. The instrument
should be relevant to the purpose of your evalu- ation.
2. Clinical utility. To be useful within the clinical setting the measure should be simple, easy to use, accessible and acceptable to the patient. A manual should be available, describing clearly and in detail the standardised procedure in which the measure should be used. Importantly, the measure should not take up more resources than are avail- able, either in terms of its cost or the time taken.
3. Scientific properties. The measure should possess three scientific properties:
• reliability: the results produced should be accurate, consistent, reliable over time, and reproducible within and between raters • validity: it should measure what it purports to
measure, in the population and within the setting in which it is being used
• responsiveness: it should be able to detect clinically important change.
4. Standardisation. It is advantageous to choose a measure that is widely known (for example the Medical Research Council muscle strength grading system). Provided that the measure is used correctly, meaningful informa- tion can then be communicated within and between different professionals and different areas of service delivery. This enables uniform
monitoring of the patient's condition over the longer term. This is particularly relevant to many neurological conditions where the person fre- quently accesses a wide variety of services over many years, and hence is reassessed by a number of different people in a diversity of settings.
Ideally, the outcome measure chosen should have been comprehensively evaluated and possess all of these attributes. Unfortunately, however, this is often not the case. To date, relatively few of the clinical measures available for measuring move- ment and function in neurological patients have been extensively and rigorously evaluated. It is the responsibility of clinicians and researchers to review critically the outcome measure chosen to determine how well it meets these criteria. This must be considered in the context of the population and the setting in which it will be used. This knowl- edge is essential to enable accurate interpretation of the information gained from the measure.
Ensuring the process of measurement is rigorous
The process of measurement must be rigorous if the results are to be credible. Measurement at the time of the initial assessment, before a treatment programme has commenced, is essential. The timing of subsequent measurements will vary according to the length of time that it is expected to take for a change to occur. To ensure reliability the assessments should be undertaken in accordance with operational guidelines and the measurement process should be clearly documented. It is im- portant that the measurements are undertaken in similar circumstances if the comparisons made between evaluations are to be valid. For example, the assessment should be consistently undertaken prior to treatment rather than post-treatment when the patient's performance may be negatively affected by fatigue, or perhaps positively affected by the treatment which has just been given.
Interpreting the results
The primary reason for using outcome measures is to evaluate the success or failure of interven- tion, in a standardised and objective manner. It is
essential that the information generated is used to inform practice. This requires the clinician to be clear about whether the outcome is attribut- able to the intervention given (Sackett et al 1996), and to reflect on and critically appraise the care given. Accurate and meaningful interpretation of the results must be based on a sound under- standing of both the clinical context (e.g. the severity of the condition, the purpose of the intervention) and the measurement process (e.g. psychometric properties of the measures used, timing of the measurement).