Objetivos Específicos

1.4.1 Australia 1.4.2 Sweden

1.4.3 Commonwealth Secretariat 1.4.4 World Bank

institutionalise school-based performance indicator systems, and relevant innovations. Hence the apparently curious choice of agencies/countries.

1.4.1 Australia

Ruby (1994) highlights six lessons that emerged from only the first twelve months of institutionalising a performance indicator system:

1. It is difficult to communicate accurately and economically about ways of assessing performance given:

• the technically and theoretically complex nature of indicators; and,

• any external assessment potentially challenges concepts of professionalism, traditional notions of autonomy and raises questions about the nature of accountability.

2. Technical problems of outcome measurement do not predominate; instead the focus is on problems of interpretation and the influence of contextual and process variables.

3. The importance of stressing fundamental questions of why there is a demand for performance indicators, and in what context and for what purposes they are useful, rather than on the technical and practical questions of constructing indicators.

4. Indicators are essentially normative and goal oriented, directly linked to policymaking and the political process, and only useful when linked to a model of the education system.

5. The importance of drawing on as many paradigms and perspectives as possible, to involve people working in science policy and public policy as well as education.

6. The benefits of exploring new ideas using a co-operative and relatively open process. Keeping the process of analysis transparent to those

affected by the outcome establishes credibility for the outcome of the process and of those involved.

While it is not feasible to incorporate all these lessons (e.g. to have an agreed model of every education system as a basis for a framework), issues of communication,

interpretation, pluralism and transparency are important to bear in mind when designing sets of performance indicators which will be acceptable within different country

contexts.

1.4.2 Sweden

The Swedish National Agency on Education set up an evaluative project in 1992 to examine the non-cognitive development of pupils in Swedish schools. The project took account of pupils' own views as 'connoisseurs of their own schools'. It examined pupils' development on four core variables which reflected strong national purposes:

independence, self-confidence, participation in decision-making and solidarity with others. This was based on the view that:

Individuals that hold a critical mind and are used to act in independent ways are seen as important parts of the assurances that the Swedish society have taken towards fascism. (Ekholm and Karang, 1993: 13) In other words, it was important to measure self-confidence as this was seen as a

prerequisite for successful learning, and involvement in decision making as essential to sustaining democracy. Tolerance and understanding of others were also seen as

essential to democracy (Ekholm and Karang, 1993: 14).

Commentary for DFID

The purpose of supporting primary education in developing countries is, in part, support of democratisation (Western style) - see ODA (1993) and DFID (1997). Thus, while simple counts of participation or registration in primary school - and, eventually, attendance at primary education - may have been sufficient at one stage, discussions about measures of outcome or success are usually in terms of achievements in literacy, numeracy and science. But, where democratisation is a central tenet of a donor's

strategy (as with the DFID), appropriate indicators of aid effectiveness are required; at a minimum, measures of school effectiveness have to include non-cognitive

achievements.

1.4.3 Commonwealth Secretariat

Davis (1996) set out to compare the progress made in developing performance

indicators at the higher education level in Commonwealth countries. In the preface to her study, Fielden writes:

"Surely, we thought, at the very least, we can obtain staff-student ratios from various different jurisdictions" (Fielden, preface Commonwealth Higher Education Management Service, 1996.

Despite the rhetoric about performance indicators, however, little was actually obtained, and so:

"Our aim is to show how very little has been achieved and how, despite the massive industry of researchers working on performance Indicators, comparatively few are in use nationally." (Fielden, op cit)

Commentary for DFID

Part of the problem was that performance indicators are used for different purposes in different contexts (see section 1.3 above). This study graphically demonstrates the gulf between the identification of a possible indicator - and even ways of collecting,

relatively cheaply, the corresponding data - and the use of an indicator to inform policy.

1.4.4 World Bank

Prior to 1987, a basic data sheet giving country economic and social and sectoral data was required for all Bank-financed education projects. This was abandoned

subsequently, as the data collected were often of poor quality and not seen as relevant to individual projects.

General experience of indicator use

The World Bank Operations Department report that there are:

• disparate educational information systems between and within countries;

• differences in educational systems;

• differences in classification and terminology; and

• imbalances in the collection of data.

They identify possible reasons as: complexity of education, lack of resources, lack of capacity to carry out educational research, the political nature of educational data and lack of standardisation of education system components (McRae, 1990).

increasingly characterised by a focus on developing local capacity for project benefit monitoring and evaluation. This shift of focus has occurred in response to the

Wapenhans Report (1990) which points to:

• too much emphasis on the mechanics of project implementation;

• poor identification of risks and factors influencing project outcomes;

• lack of objective criteria, transparency and consistency across units; and

• ratings which tend to be overly optimistic.

For the purposes of the Bank and its clients, the most significant benefits of

performance indicators accrue in project design, project supervision and monitoring and project evaluation (World Bank, 1996).

The approach is spelt out in a paper by Sigurdsson and Schweitzer (1994). This paper discusses three types of data: basic data (providing socio-economic background and context), education sector data (useful in project identification and evaluation), and project performance data (to mark the progress of project components towards specific targets).

The appropriate types of indicators related to the project cycle are: input indicators; process indicators (to monitor stages of project implementation); output indicators are the immediate project targets identified as project components to be completed; and impact indicators are derived from sectoral data (See Annex 1A for further details). The World Bank suggests that: "Policy related indicators can be used to identify risk and enabling factors during preparation and appraisal for projects and systems."

(Sigurdsson and Schweitzer, Executive Summary, 1994). It is recognised, however, that to be meaningful, education indicators must be analysed in the context of system needs and available financing. The danger that funding decisions based on indicator

performance may encourage skewed or falsified data recording is also acknowledged (1994: 3).

Sigurdsson and Schweitzer conclude with the following recommendations:

• that project performance indicators be project specific;

• that a uniform approach to economic justification be applied;

• that the World Bank should assist UNESCO in their ongoing work to define desirable definitions for MIS systems in Bank-financed projects.

Commentary for DFID

Many of the lessons from the World Bank's experience and some of its

recommendations should be adopted. Disregarding the Bank's obsession with a uniform economic justification of projects, the emphasis on project-specific performance

indicators, the importance of process as well as outcome indicators and the need to pay careful attention to the source data, are all sensible points.

1.5. SPECIFICITY OF EDUCATIONAL

SYSTEMS IN DEVELOPING COUNTRIES