In order to establish the relevance of the theory of measurement to the measurement of health outcomes, the characteristics of an outcome must be presented. The problem for physiotherapists and other members of the professional allied to medicine lies in the conflict that appears to exist between their own definitions and subsequent methods of evaluating health outcomes and those of medical colleagues. The UK Clearing House for health outcomes at the Nuffield Institute for Health, University of Leeds, was set up by the NHS Management Executive in association with the Royal College of Physicians. The purpose of the clearing house has been extensive, one of the aims being to 'raise awareness about key issues in health outcome measurement, in particular the issue o f
attribution.' (Outcomes Briefing 1994).
In a document on health outcomes produced by the Clearing House (1993) they define health outcomes as being:
'Outcomes, in general, are the results (effects) o f processes. They are
that part o f the situation pertaining after a process which can be
attributed to the process.'
Clearing House, Health Outcomes University of Leeds (1993) This leads the authors to therefore deduce that : 'The irreducible pari o f this definition is that outcomes are effects. Measurement which cannot estimate effects is not strictly
outcome measurement.' (1993)
Unfortunately these definitions then lead them to an assumption that in order to measure the effect or efficacy of intervention, only two approaches are available to the health care professionals. These are as stated, the randomised controlled trial and the post-hoc multi variate correlation and analysis of variance techniques .
This medical concept of health care measure is in direct conflict with that of the professional allied to medicine. Lohr (1988) suggests quite a different concept of outcome measurement more relevant to physiotherapists and their colleagues. He suggests that the Donabedian definition of outcome, ‘direct attention specifically to the
patients well-being \ and ‘emphasises individual over groups ’ is far more relevant to the measurement of outcome (Lohr 1988). Benjamin (1995) takes this a step further by suggesting that Lohr’s interpretation of Donabedian's definition necessitates studies that look at individuals response examining a broad range of outcomes. She further suggests that this concept clearly requires studies that identify the effectiveness of input as opposed to the medical model of studies examining the efficacy of treatments.
Whilst these terms are frequently misunderstood and used interchangeably, their correct definition is crucial to stroke evaluation research. Lohr defines efficacy as being: 'The level o f benefit experienced when health care sennces are applied under 'ideal'
conditions'. (1988) That is the randomised controlled trial situation. However, he
defines effectiveness as being: ‘The level o f benefit when sennces are rendered under
ordinary circumstances by average practitioners for typical patients'. (1988). Benjamin
supports this argument by highlighting the different aspects of what she terms effectiveness research to include quasi-experimental designs, research that emphasises clinical practice and not focused on a narrow range of clinical end points. It is interesting that she suggests that this is what is required by policy makers and service purchaser, not the efficacy research of the randomised controlled trial:
'Policy application may flow more directly fi'om effectiveness research. The emphasis on real practices, providers, and patients leads to greater generalisation o f results. The fact that treatments are provided under typical conditions means that cost data Detective o f actual practice can be collected and utilised in cost-effectiveness analysis attd other cost- based comparisons.'
Benjamin (1995) It is interesting that Benjamin feels that medical input is often associated with the treatment of pathology whilst the therapist's role is focused on the ‘elimination or
amelioration o f impairment and disability' (1995) improving patient's functional status
and quality of life.
It can therefore be argued that the outcome of stroke rehabilitation is the clinical effectiveness of the process being implemented. Therefore outcome measurement should
be of changes in function and behavioural status, occurring in a clinical setting during physiotherapy interventions in the environment most relevant to the individual patient. 3.2 Clinical and Statistical Significance
The previous identification of the characteristics of health outcomes was inevitably from a clinician's perspective. However health service administrators require group data in order to identify global management issues. When group data is provided from RCT, problems identified cannot be resolved, as the individual's data is lost in the statistical process of analysis. However, where group data is provided from individual case studies advocated by clinicians, any global problems identified can be resolved by reverting back to the individual from the group. In order to fully understand this concept, the term clinical significance and statistical significance must be identified and reviewed.
The term clinical significance is frequently seen in the research literature when the presence or absence of statistical significance is discussed in the light of statistical findings within a group. As Hicks (1995) discusses, any research reporting the statistical significance of data must be critically reviewed to establish whether the patient has benefited clinically. In a group of post-partum women undergoing two different pelvic floor exercise regimes she writes:
'Let us imagine at the end o f the study that this latter group was found to have pelvic floor muscles which were significantly stronger statistically, but yet they still suffered a high level o f incontinence. It could be said that these results were statistically significant but were clinically meaningless.'
Hicks(1995) This important distinction is also identified by Altman (1995) in his book on statistics for medical research in which he cites Frieman et al., (1978) who looked at 71 published papers with negative results defined as having p values greater than 0.1. Frieman constructed confidence intervals for each study and found that for half the studies the results were compatible with 50% therapeutic improvement which, as Altman suggests, may <reasonably be taken as clinically valuable \ He goes on to state that smaller
studies may also fail to detect as statistically significant a difference that is real to the patient. ‘These trials demonstrate the non-equivalence o f statistical significance and
clinical importance. ’ The reason for this goes back to the previous discussion regarding
the definition and characteristics of an outcome. Most medical researchers use a reference point for their statistical calculations of ’cure’. A more appropriate outcome, previously established, should be clinically significant change that is, improvement or lack of improvement in a name function or behaviour.
In Testa's paper on the nature of outcomes Assessment in Speech Language Pathology (1995) she discusses the definitions of clinical significance and the relationship between these definitions and research methodology. In her paper she cites Bain's (1991) definition as being:
'a clinician's subjective judgement o f the importance o f the change observed in a client who is undergoing treatment.'
Bain et al., (1991) A more objective definition of clinical significance is offered by McReynolds and Kearns (1983) as being:
'the strength o f treatment effects, ’ that is, the clinical or therapeutic relevance o f treatment '
Kearns and McReynolds (1983) From these definitions it can be seen that there are two fundamental components of clinical significance. On the one hand the clinicians judgement of the importance of the observed change, and on the other the relevance or relationship of that change to the intervention. Judgement by clinicians must be based on knowledge and based on that knowledge, change that requires therapeutic input is clinically significant.
The function of statistical significance is to show whether there is enough evidence to draw a conclusion about the differences observed between groups. Is the difference larger than the random difference created by the sample selection? Whilst this method may demonstrate that on average the difference is a real difference, there is no way of
knowing from the group which patients have benefited from the treatment, and which have not. It could therefore be argued that statistical significance may bear no relevance to the effectiveness of interventions for any given individual and that clinically significant change if based on sound theoretical knowledge is a more appropriate reflection of effective intervention and should therefore be used as a measure of health outcomes. Similarly it might be suggested that the use of group measurement such as averages is limited and whilst it may provide information for administrators, further problem solving processes are hindered by an inability to revert the group data back to the individual. This in contrast to the measurement of clinically significant functional or behavioural change occurring in the individual that can be used, either, as a clinical measure of effectiveness or to provide useful information about groups of patients for effective management of resources.