LA ESTRUCTURA Y MARCOS DE LA ACCIÓN COLECTIVA

Respondent engagement

Patient-reported outcome measures are unusual compared with almost all other health indicators, in that they require active input from the patient or respondent. For example, in the NHS national PROMs programme, whereas pre-operative recruitment rates to PROMs for patients receiving joint replacement surgery have been 68%, the rate is markedly lower, 41%, in patients receiving varicose vein repair surgery, although this is largely attributable to eligible patients not being invited.32 _{In a sample of patients recruited}

to complete PROMs for any of six long-term conditions recruited via general practices, the recruitment rate was lower again, at 38%.33 _{There is some evidence that response rates have deteriorated over time; Hazell} et al.34 _{found marked deteriorations in response rate to an identical survey to patients about their asthma,}

pATIENT-REpORTED OuTCOmE mEASuRES AND THE EVALuATION OF SERVICES

62

NIHR Journals Library www.journalslibrary.nihr.ac.uk

Clearly, a number of factors may influence the response rate. In the examples just cited, patients were more likely to respond to surveys about receiving a specific surgical intervention than to primary care surveys about long-term conditions where there was no link to receiving treatment. The concern is that the lower the response rate, the harder it is to use the outcomes obtained to evaluate services.

However, the greater concern is if there is evidence of response bias: if important characteristics of either patients or services are associated with differential response rate. There is evidence that this may occur for PROMs. In the national PROMs programme, Hutchings et al.35 _{found that the poorer the health status}

in terms of comorbidities prior to surgery, the poorer the response rate to PROMs questionnaires mailed out after surgery. Similarly, in the primary care study of PROMs for long-term conditions referred to above, when patients were asked to return a follow-up PROM 1 year after the baseline, the response rate was lower in those with poorer baseline health status.36 _{It is clear that such biases may produce misleading}

evidence of the effectiveness of services.

There are more subtle forms of problem with PROMs if questionnaires are returned incomplete, for example if more sensitive or personal questions are not completed or more difficult items are omitted. Usually the development phase of PROMs reduces such risks. There is a range of statistical techniques for addressing missing or incomplete data from longitudinal data sets such as PROMs, most commonly the use of imputation methods derived from the information about respondents that has been obtained. Analysing missing data from the national PROMs programme, Gomes et al.37 _{found that inferences about the}

performance of services could be influenced by the assumptions and methods made to make imputations to address missing data.

To address the problem of non-response, most effort has gone into the design of PROMs and mechanisms of delivery (e.g. the use of the internet and other modern media), partly because these are practical solutions that can be implemented. The scope for innovative technology to enhance respondent

engagement is significant, although, currently, for the majority of PROMs, evaluative research still relies on traditional survey methods, particularly the mailed questionnaire. Although beyond the scope of this essay, recent innovations in the core format of PROMs may also strengthen their acceptability and power to engage the respondent. The innovation is the use of computer adaptive testing (CAT) to tailor

questionnaire items to the individual, with responses to initially administered items determining the choice of subsequent items. The total number of items required to be completed is significantly reduced. This has been the subject of a major National Institutes of Health-funded initiative in the USA, Patient Reported Outcome Measures System (PROMIS) (www.nihpromis.org/), to produce questionnaire items that can populate CAT systems. Although an exciting initiative in the science of PROMs, particularly in terms of their use in assessing individuals’ health, there is little evidence to date of their use in evaluative research. The other major concern occurs where there are major difficulties in engaging respondents, whether because of major physical, cognitive or developmental limitations or because of social exclusion. One example is the involvement of children. PROMs exist that are designed to be relevant to the perceptions of children.38 _{Additional care may be required, in terms of appropriate interviewing, determining intellectual}

capacity, not relying solely on chronological age, and the role of observation and proxy informants.39

Dementia poses related challenges. There are dementia-specific PROMs, which appear to work

satisfactorily for patients with mild to moderate dementia.40 _{However, for those with more severe levels of}

dementia, assessments by a proxy carer are the more plausible option. Nevertheless, proxy ratings should not be treated simplistically or uncritically because their rates may, in turn, be influenced by burden of care and carer burnout, and appropriate adjustments are, therefore, required.41

In some areas of health care it is impossible to obtain a pretreatment or baseline assessment for obvious reasons; patients who experience sudden health events such as a stroke or hip fracture will not have any reason to have completed such an assessment. A number of approaches may be adopted for which the patient’s perspective is needed. One approach is simply to give up on pre-event assessment and assess

progress and possible impact of interventions after the sudden event and as soon as it is feasible to engage the patient.42 _{Alternative strategies can include inviting the patient retrospectively to assess their}

pre-event health or to use appropriately matched population-based data. Neither of these two strategies is straightforward.

There is only limited evidence of the impact of social exclusion in relation to PROMs. The problem is, however, well highlighted in a study by Jahagirdar et al.,43 _{who found that excluded groups, such as those}

with learning difficulties or low literacy, were less likely to be involved in the development of PROMs for chronic obstructive pulmonary disease. The commonest pragmatic solution to social exclusion more generally is to weight evidence for under-represented groups in surveys. A recent review of the sparse evidence argues that multiple and flexible approaches are needed properly to ensure that socially excluded groups’ views are not overlooked44 _{(www.pssru.ac.uk/archive/pdf/4390.pdf). The importance of social}

inequalities and social exclusion in health services research is further discussed in Essay 5 in this volume.

Health professional engagement

If the role of PROMs is to be expanded, it is likely to require greater engagement of clinicians so that collection of PROMs becomes more a part of routine care. Reference has already been made to evidence from trials introducing PROMs into individual patient care and showing mixed evidence of impact on management decisions, patient experience of care and ultimate health outcomes. This variable impact may, in turn, be attributable to health professionals’ attitudes, beliefs and experiences regarding PROMs; these have been the subject of some research. A recent study of surgeons’ views of PROMs found considerable diversity with a range from enthusiast to sceptic.45 _{In this and other studies, a number of concerns have}

been expressed. Cognitive problems include the view that PROMs data are difficult to interpret and relate to management decisions.46,47 _{Other concerns focus on logistics, time constraints and difficulties}

of incorporating PROMs into clinical routines.46–48 _{These studies generally stress clinicians’ expressed}

need for greater training to incorporate PROMs into practice.49 _{At worst, some studies suggest concerns}

that PROMs actually may cause harm if evidence from them is misunderstood or misused by third-party audiences such as managers, commissioners or politicians.50,51

Patient-reported outcome measures and routine health-care revisited

A recent survey obtained the views of relevant experts from the USA, England and the Netherlands about prospects for future use and impact of PROMs.52 _{There was a clear consensus that for PROMs to have}

their fullest impact in assessing the performance of health services, there needed to be greater integration of information systems for routine patient care and for system performance measurement. Currently in all three countries, information systems for the two different functions are effectively independent. The experts identified a number of barriers that would need to be overcome to integrate these two worlds of activity, including lack of trust from participants and insufficient belief in the value of PROMs in patient care.

A number of reports are beginning to appear broadly supportive of the conclusions of Van der Wees

et al.52 _{and also express optimism that the integration of information from PROMS for patient care and}

system performance can be achieved. In the USA, Wu et al.53 _{describe a number of health record systems}

already available and in use to support the two functions. They identify three developments favourable to this integration: the positive trend towards patient-centeredness and electronic applications for PROMs, the growth of electronic health records and trend towards comparative effectiveness research that is patient oriented. More recently, Jensen et al.48 _{reported a number of encouraging case studies in the USA}

in which PROMs served both patient care and system evaluation. They effectively identify the same three generally supportive developments as Wu et al.53

Warrington et al.54 _{provide a positive account of a NHS setting in which PROMs have been successfully}

implemented to provide long-term follow-up of cancer survivors. They acknowledge that in a systematic review of their field of cancer care, Nama et al.55 _{could find no high-quality evaluations to demonstrate}

pATIENT-REpORTED OuTCOmE mEASuRES AND THE EVALuATION OF SERVICES

64

NIHR Journals Library www.journalslibrary.nihr.ac.uk