Lo virtual

There are currently no systematic reviews examining the feedback of aggregate PROMs data to improve patient care. Boyce and Browne’s24_{systematic review examined PROMs feedback in the care of individual}

patients and at the aggregate level. They found only one study25_{examining the use of aggregate PROMs}

data, in which physicians were randomised either to receive peer comparison feedback on the functioning of older patients in their care or to be told that the functioning of their older patients would be monitored. This study found no statistically significant differences in patient functional status between patients in the intervention and control groups. There are four reviews of the feedback of performance data.19,26–28In general, these reviews found a small decline in mortality following public reporting after controlling for trends in a reduction of mortality; however, individual studies varied in their findings. For example, studies examining the impact of cardiac public reporting programmes on mortality rates found a variable picture: eight studies found a decrease in mortality rates over time,29–36while another four studies37–40found no changes in mortality rates over time. Similarly, although most studies examining the impact of public reporting on process indicators found an improvement in hospital quality, this varied from a‘slight’ improvement to a‘significant’improvement. However, they also found little evidence that the public reporting of performance data stimulated changes in hospitals’market share, suggesting that patients may not change hospitals in response to the public reporting of quality data. We consider these reviews in more detail inChapter 4of this report.

There are 16 reviews of the quantitative/randomised controlled trial (RCT) literature on the feedback of individual-level PROMs data24,41–56and one review currently in progress.57_{There is also one review of}

qualitative studies58_{and four mixed-method reviews.}59–62_{Thus, there are a total of 21 existing reviews}

examining the feedback of individual PROMs data in patient care. Of these reviews, one focused on

screening for mental health problems51_{in primary and secondary care, and four others focused on the use of}

oncology settings,42,43,46,55_{one review focused on use of PROMs as a means of screening for cancer-related}

distress56_{and two reviews focused on the use of PROMs feedback in palliative care settings.}59,60_{One review}

focused on feedback of PROMs data to allied health professionals.61_{Three studies attempted to identify the}

‘barriers and facilitators’to PROMs feedback in clinical practice;58,59,61_{two reviews adopted a theory-driven}

approach to the review;60,62_{and one review combined a‘review of reviews’with existing conceptual}

frameworks of PROMs feedback, but focused on synthesising the quantitative evidence.43

Thus, there is a large volume of literature examining the impact of individual PROMs feedback in clinical practice, and reviewers have dissected and grouped the literature in a number of different ways, for example by condition or by setting. Furthermore, even for those reviews that have focused on the same condition or setting, differences in search methods and in inclusion and exclusion criteria have resulted in these reviews including overlapping, but different, groups of studies. For example, both Chenet al.43

and Kotronoulaset al.42_{examined the impact of PROMs feedback in oncology settings, and both reviews}

included both RCTs and quasi-experimental studies. Chenet al.43_{identified 27 eligible studies and}

Kotronoulaset al.42_{identified 24 eligible studies, reported in 26 papers. However, only 16 papers are}

common to both reviews; 10 papers appear only in Chenet al.43_{and nine papers appear only in}

Kotronoulaset al.42_{The reviews also vary in their synthesis methodology; most adopted a narrative}

overview, but those with a more narrowly focused review question used a meta-analysis.47,51_However,

although a range of synthesis methods has been used, the reviews are dominated by traditional systematic reviews of RCTs.

It is not our intention to provide a detailed analysis of the findings of each review. Here we present a brief overview of their findings in order to highlight outcome patterns that will be explored during our synthesis. Those adopting a traditional systematic review methodology to survey the entire literature have, in general, found it difficult to reach firm conclusions about the impact of PROMs feedback on the process and outcomes of patient care, largely owing to the heterogeneity of the intervention itself, and the wide range of indicators used to assess its impact.48_{There is some evidence to suggest that the purpose or function of}

PROMs feedback may influence its impact, with greater impact on patient outcomes when PROMs are used to monitor patient progress over time in specific disease populations, rather than as a screening tool.24_{One common pattern evident in these reviews is that PROMs feedback has a greater impact on}

clinician–patient communication, the provision of advice or counselling and the detection of problems than on patient management and subsequent patient outcomes.48,49

This general conclusion is also mirrored in the reviews focusing on oncology.42,43_{For example, Chenet al.}43

found‘strong’evidence that the feedback of PROMs data improves patient–clinician communication, and ‘some’evidence that it improves the monitoring of treatment response and the detection of patients’ problems. However, they found‘weak but positive evidence’that PROMs feedback leads to changes in patient management, and‘a great degree of uncertainty’regarding whether or not PROMs feedback improves patient outcomes. Chenet al.43_{suggested that greater impact of PROMs feedback may be found}

where PROMs are fed back for a sustained period of time to multiple stakeholders, with feedback that is clear and easy to understand, and sufficient training for health professionals. Kotronoulaset al.42_found

significant increases in the frequency of discussions‘pertinent to patient outcomes’, but little impact on referrals or clinical actions in response to PROMs data. This suggests that there may be a‘blockage’ between the identification and discussion of the issues raised by PROMs and the ways in which clinicians respond to these issues.

The review of qualitative evidence58_{provides some further possible explanations for these findings, which}

can be explored in our synthesis. This review found that clinicians sometimes questioned the validity of PROMs data, and expressed concerns about the lack of clarity regarding whether PROMs data were intended for use to inform clinical care or to monitor the quality of the service. PROMs feedback was more likely to inform patient management when it provided new information to clinicians. This review also identified a number of unintended consequences of PROMs feedback. In line with some of the theories we discuss inChapter 8, the intrusive nature of incorporating discussion of PROMs data into the consultation

was, in some circumstances, perceived to affect the patient–health-care practitioner interaction. They found some evidence that, rather than open up the consultation, PROMs feedback may narrow its focus, and that certain questions may distress patients and, thus, damage the patient–health-care practitioner relationship. Thus, evaluating and reviewing the evidence of PROMs feedback is a challenge for several reasons, all of which arise from the complexity of the intervention. First, PROMs feedback is unavoidably heterogeneous and varies by PROM used, the purpose of the feedback, the patient population, the setting, the format and timing of feedback, the recipients of the information and the level of aggregation of the data.12

Therefore, there is a need for review methods that explicitly take into account the heterogeneity of the intervention, and seek to understand how this shapes intervention success.

Second, the implementation chain from feedback to improvement has many intermediate steps and may only be as strong as its weakest link.62_{At an individual level, PROMs feedback may improve communication}

and detection of patient problems, but may have less impact on patient management or health status.48

However, its impact on communication during the consultation is not uniform and depends on the nature of patients’problems. In oncology, where there is most evidence that PROMs influence communication, clinicians were more likely to discuss symptoms with their patients in response to PROMs feedback, but not psychosocial issues.63,64_{We are confronted with the cautionary hypothesis that PROMs feedback may not}

result in further discussion or the offer of symptomatic treatment because high PROMs scores (suggesting high disease impact) do not always represent a problem for the patient or a problem that clinicians perceive as falling within their remit to address.65

At an aggregate level, there are many organisational, methodological and logistical challenges to the collation, interpretation and then utilisation of PROMs data.66_{These include reducing the risk of selection}

bias, as older, sicker patients are less likely to complete PROMs;67_{reducing the variation in recruitment}

rates in PROMs data collection across NHS trusts;68_{ensuring that procedures are in place to adequately}

adjust for case mix;69,70_{collecting the data at the right point in the patient’s pathway; and summarising this}

information in a way that is interpretable to different audiences.71_{In summary, a number of potential}

obstacles may prevent or lead to partial success in PROMs feedback achieving its intended outcome of improving patient care. There is a need to pinpoint these obstacles or blockages more systematically in terms of their location in the implementation chain, and to identify the circumstances in which they occur and those in which they can be overcome.

Third, the success of PROMs feedback is context dependent, and these contextual differences influence the precise mechanisms through which it works and, thus, its impact on patient care. For example, using PROMs data as an indicator of service quality for surgical interventions in acute care is very different from their use as a quality indicator of GPs’management of long-term conditions in primary care. The impact of surgery on disease-specific PROMs and knowledge of the natural variability of scores has been well documented,72_{but this knowledge is lacking regarding the impact of primary care on PROM scores.}73

At an individual level, surgeons are specialised and need only interpret the PROMs data in their specialty. In contrast, GPs manage patients with different long-term conditions, and need to make sense of data from different PROMs, or to disentangle the impact of different conditions on PROMs scores. The interpretation of the meaning of changes is, therefore, very different in each context.

Furthermore, differences in context can result in the intervention not working through the intended mechanisms, leading to unintended consequences.74_{For example, the feedback and public release of}

performance data may stimulate improvement activity at hospital level through increased the involvement of leadership or a refocusing of organisational priorities,75_{but it has also been shown to lower morale, and}

may focus attention on what is measured to the exclusion of other areas.18_{Others have cautioned that it}

may also lead to surgeons refusing to treat the sickest patients to avoid poor outcomes and lower publicly reported ratings.74_{Data from the national PROMs programme have been misinterpreted by some as}

indicating that a significant proportion of varicose vein, hernia and hip and knee replacement should not take place.76_{Public reporting of performance data may not improve patient care, as intended, through}

informing patient choice.19,77_{Rather, patients are often ambivalent about performance data and rely on}

their GP’s opinion when choosing a hospital.78,79_{Thus, there is need to highlight the potential unintended}

consequences of PROMs feedback and to distinguish between the circumstances in which they arise. Fourth, PROMs have been implemented against a backdrop of other initiatives designed to drive up the quality of patient care, which can potentially either support or derail the intended impact of PROMs feedback. For example, Quality and Outcomes Framework (QOF) payments are dependent on the use of a standardised questionnaire for depression screening, resulting in GPs sometimes avoiding coding a person as suffering from depression in order to circumvent the completion of a questionnaire viewed by many GPs as unnecessary.80,81

Finally, despite PROMs feedback having many functions and aspirations, research coverage of them is uneven, with more studies (trials and qualitative case studies) examining PROMs feedback at an individual level and few studies examining their use as a performance indicator at a group level.