Técnicas de procesamiento de análisis de datos

a package so that

each party making

a commitment sees

that others are also

contributing

Improving Lives through Impact Evaluation

many different forms, from an interagency committee to a network, secre- tariat, or independent organization.15 _{The choice will depend on assessing} the relevant tradeoffs, and the institutional structure should ultimately be guided by the structure that will best fulfill a range of aims, including high standards of technical quality, independence and legitimacy, operational effi- ciency, international leadership, and mobilization of additional resources for impact evaluation. Some ideas are presented schematically in table 3.

High standards of technical quality. The greatest risk to this initiative is that

it might mobilize additional resources for impact evaluation and end up financing studies that fail to generate strong evidence. Institutional designs that give the council greater autonomy and involve evaluation experts in its governance are more likely to set and maintain high standards. Institu- tional designs that engage members in negotiating and setting standards

Table 3 Implementation options

Interagency committee Special program within an existing organization Secretariat

Structural characteristics Governance Members appoint staff to act as

liaison Members elect a supervisory com-mittee Members elect a board

Resources required from

members Mainly staff time Staff time and funds Staff time and funds Funding for core functions None Membership dues Membership dues

Funding for impact evaluations Members participate in funding activities on the basis of voluntary independent commitments that they manage themselves

Members make commitments to spend a specific share of their social program budget, grants, or loans on their own impact evaluations or impact evaluations commissioned by the council

Members make commitments to spend a specific share of their social program budget, grants, or loans on their own impact evaluations or impact evaluations commissioned by the council

Staffing No specialized staff Staff dedicated to managing techni-

cal review and support Staff dedicated to managing techni-cal review and support and some administrative functions

Tradeoffs

Direct costs Lowest Medium Highest

Indirect costs Relies on borrowed staff for techni- cal, financial, and administrative functions

Relies on borrowed staff for financial

and administrative functions Lowest; staff time required to par-ticipate in committees, reviews, and governance

High standards of technical quality Difficult; least agile decisionmaking structure and limited autonomy and engagement of technical experts

Moderate difficulty; focused manage- rial attention but limited autonomy and engagement of technical experts

Least difficult due to focused mana- gerial attention and dedicated techni- cal experts

Independence and legitimacy Low Low High

International leadership Middle Low High

Operational efficiency Low cost but correspondingly low output; depends critically on effi- ciency of coordination mechanisms and fulfillment of commitments by members

Moderate costs but commensurately larger output; depends critically on efficiency of host organization and dynamic between members and host organization

Greater direct costs but correspond- ingly greater output; depends critically on scale economies and coordination with members

Ability to mobilize additional funds Moderate, depending on how engaged members are and how actively they focus on the initiative

Low, depending on how high a prior- ity is given to the initiative within the host organization

Moderate to high, depending on engagement of members in policy decisions and demonstration of the initiative’s value to stakeholders

When Will We Ever Learn?

are more likely to achieve full collaboration in supporting and following such standards.

Independence and legitimacy. The legitimacy of impact evaluations

depends, ultimately, on their rigor. If studies are done well, then their find- ings will be recognized as legitimate through any expert review—whether for publication in an academic journal or inclusion in a systematic review (Campbell Collaboration 2005a, b). However, impact evaluations play a role in spheres of debate that do not necessarily involve expert reviews. In particular, when findings are debated in the press, in civil society, and in government, technical quality is not necessarily self-evident. Other signals may be used to assess a study’s legitimacy—such as how it was financed, who conducted it, and how it was approved or published. Institutional designs that provide the council with greater autonomy will benefit the initiative to the extent that the council develops a reputation for inde- pendence and integrity. Members will also benefit when they can rely on the council’s legitimacy to review and independently validate any impact evaluations they have conducted.

Operational efficiency. The council’s operational efficiency will be enhanced

by exploiting economies of scale and scope and by avoiding duplication of functions. An institutional design that takes advantage of administrative capacities in member organizations or networks is likely to be less costly and more productive than one that establishes new administrative capaci- ties.16 _{Institutional designs that rely on existing activities and structures} rather than duplication may also be more efficient whenever the marginal costs of assuming additional responsibilities are low.

International leadership. The initiative will be more successful to the

extent that it promotes a wider appreciation of the value of rigorous impact evaluations. Being a collaboration of pioneers—committed developing countries, international agencies, foundations, NGOs, and research centers—the initiative will have a natural platform for educating the public, promoting rigorous standards of evidence, and encourag- ing better use of information in policymaking. The council’s ability to lead will be affected by its identification with member organizations, its perceived independence and integrity, and the scale of its capacity to assume new tasks and be proactive. Institutional designs that increase the involvement of members in the council’s operation can enhance the council’s standing when it takes on leadership tasks; however, such involvement can also reduce the council’s agility and flexibility, its capac- ity to develop focused messages, and its ability to respond quickly to new opportunities.

Will we really know more in 10 years?

Imagining 10 years into the future, when the target date for the Millennium Development Goals has come and gone, the international community could be in one of two situations.

Members will also