Retos en la protección de microrredes

The researcher questioned key informants regarding their perspectives on setting targets for publicly reported measures, including: whether Ontario should set performance improvement targets; who should set targets; how targets should be set; and how providers should be compared, i.e., to each other, to standards, to an average/mean, or to their previous performance. Feedback from interviewees was very similar to the feedback on incentives for improvement and centred on the following sub- themes: targets are important, there are challenges to setting targets appropriately, and there is a lack of accountability in the system for achieving targets. Additionally, several interviewees provided recommendations for more effective ways to develop and use targets.

No key informant commented that targets are not appropriate for public reporting. While generally interviewees commented on the value of targets, at least one third of the interviewees shared the perspective that setting targets is, in fact, essential to driving improvement. All interviewees

121

commented on the challenges of setting targets, with the cited challenges including: caution on the potential for interrelationships in measures, in which setting a target in one area could have an unintended negative impact in another area; setting targets only where they add value; setting targets based on evidence; differentiating “stretch” targets from “hard” targets; and communicating the rationale for chosen targets. This latter comment was made in reference to the difficulty for

governments in communicating the difference between realistic short-term targets and longer-term aspirational targets to a public audience.

The following are sample comments on the challenges of setting targets and were provided by two interviewees from agencies that produce public reports.

“One of the things that has driven me absolutely bananas is people suggesting standard 5%, 10% improvement on indicators, and obviously, depending on the indicator sensitivity, that may or may not be possible, so one of the things that we were talking about was, sure for hand- hygiene, if you’re at 78% you can move it, but one of the things we know that for patient experience, if you moved it 1% or 2%, that's amazing, that’s a lot. You’re not going to see a 5% improvement in patient experience.” (comment from reporting agency)

and

“…to think about what are the unintended consequences of setting targets and there’s noise about that---and the wait times literature about when we did all the hips and knees---maybe we cannibalized all the ankles (negatively impacted wait times for ankle surgery).” (comment from reporting agency)

Two providers specifically commented on the cost and resource implications of trying to reach targets.

“If there is a value that something happens at the 10 day mark versus the 30 day mark, then a target to improve performance is a valuable thing. But if it doesn’t matter, should we be thinking sometimes the other way, such as how much is it costing you to get services on day four and is that reasonable? And could we redistribute these funds to where there are more challenges?” (comment from provider)

Interviewees from all groups provided recommendations for improving the target setting process, including: engaging patients in setting targets; having a transparent process; balancing stretch and realistic targets; acknowledging providers that show improvement, rather than focusing only on

122

achieving targets; using evidence, benchmarking, and international comparators; and understanding variation and the impact of regional issues on performance before setting targets.

“The best, of course, is when you use empirical evidence to see what the benchmark situation is… So a target should be realistic; it should be empirically evident. There should be a benchmark evident that it’s already been reached. If that’s not the case, you don’t really know.” (comment from an “other stakeholder”)

and

“To go back to the example of access to primary care to GP (general practitioner) within 48 hours, you know we are the worst in the world and we don't have any jurisdiction that is doing better. If I use this criteria of top 25%, I’m going to have a couple of jurisdictions that would be looking good and would be considered as good performers when really there is no good performer in this country. So I think the more sound targets we can develop based on clinical benchmarks and clear expectations derived from engagement with, frankly, the public and patients, I think we would be better off.” (comment from reporting agency)

Finally, interviewees from the other stakeholder group were more likely to comment on the need for greater accountability for providers to achieve, or at least move towards, targets.

“I think they're a good thing because that’s something to aspire towards, so it’s a positive and if you are looking at your own figures, and you are like ‘Oh, wow!’. I would hope every provider internally would reflect at some point on ‘where do I measure up, and hopefully I'm above the norm’. I think everyone would want to aspire to that, but honestly they’re so busy and who is going to hold them accountable?” (comment from ‘other stakeholder’)

Table 19 summarizes the sub-themes that emerged under the topic of target-setting. The strongest areas of congruence across all groups were that targets are important, that there are challenges to setting appropriate targets, and that there are opportunities to improve how targets are set in Ontario.

123 Table 19: Summary Analysis of Setting Targets Theme

Sub-themes derived from interviews Interviewee grouping

(‘x’s indicates theme identified by groups below) Reporting agency/funder Health system provider Other stakeholders

Targets are important x x x

There are challenges to setting appropriate targets

x x x

Lack of accountability for achieving targets - - x

There are opportunities to improve target setting

x x x

Grey shading indicates highest degree of accord across the different key informant groups.