A number of studies have examined public-sector entities‟ service performance information during the reform periods in various countries, for example, the United Kingdom (Banks, Fisher, & Nelson, 1997; Boyne & Law, 1991; Hyndman & Anderson, 1995, 1998; Pendlebury, Jones, & Karbhari, 1994; 2000), Australia (Herawaty & Hoque, 2007; Kloot, 2009; Lee, 2006; Nelson, Tower, Banks, & Fisher, 1997; Ryan, Stanley, et al., 2002), and New Zealand (Alves, et al., 2005; Coy, Dixon, Buchanan, & Tower, 1997; Coy, et al., 1994; K. Dixon, et al., 1991;
Lonti & Gregory, 2007; Smith & Coy, 2000; Thompson, 1995; Tooley & Guthrie, 2001; Wei, Davy, & Coy, 2008). Regardless of the different research criteria used
84
in the examinations, public-sector entities in the United Kingdom, Australia, and New Zealand were in general seen as under-performing in their service performance reporting.
Given non-mandatory service performance reporting, studies of performance reporting in the United Kingdom public-sector focused on performance indicators/performance measures and showed that performance reporting improved after the New Public Management reforms, but remained disappointing (Banks, et al., 1997; Boyne & Law, 1991; Hyndman & Anderson, 1995, 1998; Pendlebury, et al., 1994). Boyne and Law (1991) investigated the performance indicators provided by 37 Welsh district councils on 13 services56 for the years 1981/1982 – 1988/1989 and found that the number of performance indicators about output, outcomes, efficiency, and effectiveness were poor, while of those about inputs were popular.
Pendlebury, Jones and Karbhari (1994) and Hyndman and Anderson (1995) analysed the performance indicators of the United Kingdom Executive Agencies (government agencies) in their 1991/1992 annual reports and found that few or no indicators were included about efficiency and effectiveness of services. Hyndman and Anderson (1998) extended their investigation of Executive Agencies‟ performance reporting in 1990/1991 to 1993/1994 annual reports and found the numbers of performance indicators increased, and that there was a move away from simple indicators of inputs and outputs towards more sophisticated measures
56 These services included highways and transport, planning and economic development,
environmental health, street cleaning, waste collection, waste disposal, housing: homelessness, housing: management and maintenance, housing: renovation grants, housing: benefits, leisure and recreation, parks and open spaces, and cemeteries.
85
of efficiency and effectiveness. However, they contended that these changes were still inadequate. Similarly, performance indicators in United Kingdom universities were found to improve over a period (1992-1994) but were still very poor. For example, none of the universities reported information about targets (Banks, et al., 1997).
Rutherford (2000) examined the presentation and characteristics of the 1996/1997 performance indicators of the United Kingdom Executive Agencies and found that performance information was prominently disclosed. However, the understandability of many performance indicators was questioned. The United Kingdom studies showed that the merit in United Kingdom public-sector service performance reporting lay in its pursuit of accountability, in spite of its being non- mandatory and the absence of accounting standards for such reports.
Many studies in Australia have examined public-sector entities‟ annual report disclosures, including performance information (Herawaty & Hoque, 2007; Kloot, 2009; Lee, 2006; Nelson, et al., 1997; Ryan, Stanley, et al., 2002). Nelson, et al. (1997) studied university annual report disclosures from 1993–1995 and concluded that performance information was not adequate and was also not improving over time. Similar results were reported by Ryan, Stanley, et al. (2002) after their study of the 1997-1999 annual report disclosures by Queensland local governments. Lee (2006) examined the 1998–2002 performance information within annual reports of Australian government trading enterprises and found there were few service quality and efficiency measures, and that their use fluctuated over the years of the study. Although most entities stated their business
86
targets, they did not discuss how or whether such targets were achieved. In contrast, Herawaty and Hoque (2007) found that performance indicators were reported at a satisfactory level by government departments across Australia in the 2005/2006 annual reports. However, for the same period, Kloot (2009) found that fire-service departments did not provide adequate performance measures.
In New Zealand, Dixon, Coy and Tower (1991) examined the annual reporting practices of the seven universities in New Zealand from 1985 to 1989, which was a period of reform and initial impetus for greater accountability. Disclosure of service performance was non-existent in 1985 although increased over the study period (by three universities). However, none of the universities provided information about targets to enable comparison with actual achievements. Coy, Tower, and Dixon (1994) expanded Dixon, et al.‟s (1991) study to the annual reports of tertiary education institutions (including polytechnics and colleges of education) from 1990–1992. They found that the service performance information was poor, lagging behind other disclosures, and continued to be a problem, as was found earlier in Dixon, et al. (1991). Ten years on from Coy, et al.‟s (1994) study, service performance reporting by universities is still weak. Alves, Dunmore & Dunstan (2005) studied the SSPs of eight universities in 2000, 2002, 2003 and found that some universities did not provide past performance or targets for comparison with actual achievements. They argued that some targets were vague and not quantitatively measurable.
In the second year of SSP reporting requirement for local government, Thompson (1995) examined 11 local authorities‟ performance indicators in SSPs, based on a
87
single section analysis of public art galleries‟1991/1992 annual report. Thompson (1995) found that the performance measures were not linked with stated objectives. Smith and Coy (2000) study 15 city councils‟ 1996/1997 and 1997/1998 annual report disclosures. In respect of service performance reporting, Smith and Coy (2000) studied 15 city councils‟ annual report disclosures for 1996/1997 and 1997/1998 years. In respect to service performance reporting, Smith and Coy‟s (2000) study covered a wide range of activities, which were divided into four separate groups of activities: roads, parking etc; water, refuse etc; arts, culture and recreation; and other. They found, overall, that performance measures were rated well (above average), however, performance measures for water and refuse activities (including wastewater services) were rated the lowest.
Lonti and Gregory (2007) conducted a longitudinal study, examining the 1992- 2002 annual reports of five government departments. They found the average number of performance indicators per output class increased substantially between 1992 and 1997, then decreased marginally by 2002. Performance indicators about input were nil, while those about efficiency were almost absent, and outcomes and effectiveness measures (comparing actual results and targets) were very few in most departments. Performance indicators disclosed were primarily about outputs. These results seem to mirror the reform agenda, where public-sector entities are held accountable for outputs.
Most research examining the SSPs of New Zealand public-sector entities tends to focus on the education sector (for example, Alves, et al., 2005; Coy, et al., 1994; K. Dixon, et al., 1991) rather than government entities. With only a few existing
88
research studies of government entities‟ SSPs, the research on central government is more up-to-date (Lonti & Gregory, 2007). However, in respect of local government, the most recent study was Smith and Coy (2000). Further, in Smith and Coy‟s (2000) study, while SSP disclosures were considered, they were one part of the whole annual report disclosures being studied, and thus the analysis of the SSP disclosures was scant.
4.2.1 Research using authoritative requirements
The research studies discussed above used a number of different criteria to examine service performance reporting. Among them, authoritative requirements or recommendations (if the disclosures are voluntary) applicable for the jurisdiction studied were the most popular measure used. For United Kingdom studies, Boyne and Law (1991) used the Government‟s code of practice, and recommendations from the Audit Commission and the National Consumer Council. Pendlebury, Jones and Karbhari (1994) used Treasury‟s (1992) guideline: A Guide to Setting Targets and Measuring Performance. Rutherford (2000) used qualitative characteristics of financial reporting information developed by accounting professions. Hyndman and Anderson (1995, 1998) used criteria from a prior study by Brace, Elkin, Robinson, and Steinberg (1980).
Australian studies by Ryan, Stanley, and Nelson (2002) based their evaluation criteria on Coy, Tower, and Dixon‟s (1994) study, which originated from Dixon, et al. (1991), and guidelines (for example, from the Municipal association of Victoria). Banks, Fisher, and Nelson (1997) also used Coy, et al.‟s (1994) method of evaluation. Kloot (2009) used common reporting benchmarks suggested by the
89
Productivity Commission.57 Lee (2006) uses existing government documents such as from the Productivity Commission, and the International Federation of Accountants.
The New Zealand study by Dixon, Coy and Tower (1991) used primarily the accounting requirements by NZICA and a prior study of Gray and Haslam (1990) to develop evaluation criteria. Thompson (1995) used Statement of Concepts for General Purpose Financial Reporting issued by NZICA (1993). Lonti and Gregory (2007) used the Public Finance Act 1989. However, Smith and Coy (2000) used a prior research by Coy, Tower, and Dixon (1994), who modified a measure of Dixon, Coy and Tower‟s (1991) study.
4.2.2 Research using accountability expectations
Although many studies recognised some of the components of accountability expectations in some ways (Hyndman & Anderson, 1995; Kloot, 2009; Lee, 2006; Ryan, Stanley, et al., 2002; Taylor & Rosair, 2000), no study to date has used all of the components of accountability expectations as their basis for examining service performance disclosures by public-sector entities. The closest study was conducted by Taylor and Rosair (2000), using an exposition of the descriptive accountability literature by Davis, Weller, and Lewis (1989) and Sinclair (1995). They developed a quantitative measure for fiduciary and managerial accountability-based information within annual report disclosures, including performance indicators, of an Australian government department. Taylor and
57
The Productivity Commission is an Australian Government body which provides independent research and advisory services on a range of economic, social and environmental issues affecting the welfare of Australians (Productivity Commission, 2011).
90
Rosair (2000) used fiduciary and managerial accountability and examined if they were reflected in annual report disclosures of Australian government departments. Arguably, fiduciary could be broken down to legality and probity accountability, and managerial accountability to efficiency and effectiveness accountability. However, their results were aggregated and could not identify whether the components of accountability expectations were reflected in the documents, and service performance information is a small part of their annual report disclosures they examined.