The over-arching context within which elective surgery waiting times were used as performance indicators in The Canberra Times was that of the overhaul of Australian public hospital funding and performance measurement. This context is described in detail in Chapter 3.2 starting on page 61.
As described in Chapter 3-2, elective surgery waiting list lengths and elective surgery waiting times, along with emergency department waiting times, were highlighted as key indicators of public hospital performance. The outcome of the first series of agreements between the Australian State governments and the Commonwealth government set out in paragraph 3 of the overarching National Partnership Agreement on the Elective Surgery Waiting List Reduction Plan (COAG Reform Council 2009) was to ‘reduce the number of Australians waiting longer than clinically recommended times for elective surgery by improving efficiency and capacity in public hospitals’. This single statement encompasses three separate outcomes: reducing the number of people waiting longer than clinically recommended; improving the efficiency of the public hospital system; and improving the capacity of the public hospital system. The implication is that the first outcome will be achieved if the second two are achieved. The agreement provided some upfront funding divided among the states and territories, $150 million to bring about an immediate reduction in people waiting longer than recommended; $150 million for system and infrastructure improvements; and funding of up to $300 million based on how well jurisdictions met a series of performance targets.
The performance targets related to numbers of elective surgeries performed and the number of patients waiting longer than clinically recommended (see Chapter 3.2.4 starting on page 70 for details of the agreement, targets and performance against targets). The interaction of the two targets attracting reward funding was complex and it would have been difficult for each jurisdiction to work out the trade-offs
between performing as many surgeries as possible and reducing the number of overdue surgeries. The performance reports for this agreement show that each jurisdiction met and exceeded its elective surgery volume targets (see Table 3-5 on page 72) and so received the reward funding. Only three of the eight jurisdictions met their target for reducing the number of patients with overdue surgery (see Table 3-7 on page 74) while the national median waiting time for elective surgery increased from 34 to 35 days. Table 6-4 below shows the changes in median waiting times for individual states between the two time periods.
Table 6-4: Changes in median waiting time for elective surgery between 2007-2008
and 2009-2010 by state/territory State 2007-2008 2009-2010 NSW 38 44 VIC 32 36 QLD 27 27 WA 31 32 SA 42 36 TAS 36 36 ACT 73 73 NT 43 44 Australia 34 35
The fact that most states met overall surgery volume targets but did not meet the targets for primary goal of the agreement shows that the links between them are far more complex that they appear on the surface the two targets were. The rise in the median waiting times was also an indicator that increasing surgery volumes was not necessarily having the desired effect on waiting lists.
A further complication it that the rate of people joining the elective surgery waiting list is not constant and seems to be affected by how long the list is at a given time. This issue was raised by the ACT Chief Minister and Health Minister in December 2011 when she said there was an ’uncanny’ surge in additions to the waiting list every time significant inroads were made [A352011].
The pressure to meet targets and receive reward funding was intense and, in the ACT, there were accusations that the Health Directorate was manipulating patients’ urgency categories to improve the waiting time data (see Chapter 5.2.5 on page 140).
172 By tying reward funding to the achievement of a particular activity measure, the COAG reform process encouraged jurisdiction to work to maximise performance according to the measure rather than according to the overall outcome. As observed by Donella Meadows back in 1998 (Meadows 1998) the feedback loop between measuring what we value and valuing what we measure is “common, inevitable, useful and full of pitfalls” (p2, Meadows 1998).
A continuing problem in the use of reward-based funding was the lack of uniform reporting standards coupled with a lack of strong penalties for manipulating data. A paper written during the reform process (Nocera 2010) looks at the early effects of using waiting list data to allocate performance funding, asking whether the process works as a tool for reform or an incentive for fraud. As well as documenting cases where data was manipulated he gave an extreme example where work practices appeared to have been altered so much in the name of improving performance that patients were being injured and in some cases dying. One of the paper’s conclusions was that public sector data fraud should be a criminal offence.
The National Health Reform Agreement – National Partnership Agreement on Improving Public Hospital Services (COAG Reform Council 2011) followed on from the National Partnership Agreement on the Elective Surgery Waiting List Reduction Plan (COAG Reform Council 2009) and its performance targets included additional constraints in an effort to counteract the perceived ‘gaming’ of the original system. The reward funding was based on the proportional reduction in the number of patients who have waited longer than the recommended time with two constraints:
• Urgency Category 1 cases were all to be seen within clinically recommended
times by the end of 2012; and
• The 10% of patients within each urgency category who have waited the
longest must have their procedures in the reporting year (COAG Reform Council 2011).
In their work on public health performance indicators Van Peursem, Pratt et al. gave three guidelines for applying performance measurement:
A balance of ordinal, nominal and ratio indicators should be produced to avoid the impression that precision has been achieved, as well as to provide a more balanced view.
The way in which they are measured, as well as the measures themselves, needs to be an open, communicated process. It may be advisable to disclose that process and the participants who engage in it.
It should be made clear that measures are an indication of a situation which may call for further enquiry. Indicators do not provide answers, they inspire questions, and this should be made clear (p 60, Van Peursem, Pratt et al. 1995).
Looking at the use of elective surgery waiting time data in the light of these
guidelines, they partially meet the first guideline in that there are ordinal indicators (ranking the performance of states and territories) and ratio indicators (median waiting times, number of surgeries etc. They also do not fully meet the second guideline. While it was possible to find public information about how the data was collected, it took weeks of work to trace all the details and synthesise them into the narrative found in Chapter 3.3. Guideline three, arguably the most important, is not met in the available public information about elective surgery waiting times.