Limitaciones de la integración La escuela inclusiva

Hulme (1997, p3) constructed a model of impact chain (see Figure 3.1) and defines ‘impact assessment’ (IA) as to “assess the difference in the values of key variables between the outcomes on ‘agents’ (individuals, enterprises, households, populations, policymakers, etc.) which have experienced an intervention against the values of those variables that would have occurred had there been no intervention”. Based on this model, the process of IA includes three steps: choosing ‘agents’ (assessment units), choosing ‘outcomes’ (assessment indicators) and assessing.

Figure 3.1 Model of impact chain (adapted from Hulme, 1997)

Common parameters used in IA of microcredit include individual, household, enterprise, and institutional environments within which microcredit agents operate

Agent Behaviours and practices

over a period of time

Agent Modified behaviours and

practices over a period of time

Outcomes for the agent and/or other agents

Modified outcomes for the agent and/or other agents

Programme intervention

The impact is the difference between outcomes

(Hulme, 1997). Most impact studies have been conducted at the household and community levels, assessing the poverty reduction potential of microcredit (for example, Coleman, 2002; Khandker, 1998); while studies conducted at the individual level seem relatively limited and are generally restricted to evaluating the empowerment impact of microcredit on women (for example, Goetz and Gupta, 1996). A comprehensive attempt has been made by USAID’s AIMS15 Project, which seeks to assess the impacts of microfinance on individual, household and community levels and produce a complete picture of the overall impacts of the microfinance programmes (Hulme, 1997).

Impact studies that use performance indicators such as high repayment rates as a proxy for increased income to measure the success of microcredit programmes in alleviating poverty have been heavily criticised. Maclsaac (1997) argues that timely loan repayment is not an accurate indicator of improved income because even when a borrower repays a loan on time, the fund of repayment is not necessarily from the income generated from credit-supported businesses. The high loan recovery rates may be attained from the social, peer, and other forms of pressure imposed on borrowers by microcredit programmes. If the borrowers must commence weekly repayment immediately after the investment is made, they will have to repay the loan from other sources (example, from family income or moneylenders) in order to maintain their good standing with microcredit lenders (Swain, 2004; Maclsaac, 1997). Maclsaac (1997) further points out that the drop-out rates may also be high with high repayment rates, indicating that the repayment figures only report the repayment ability of those who remain in the programme. Maclsaac (1997) concludes that no direct correlation can be found between repayment and business success (improved income), and even less so between repayment and impacts on social and gender relations.

Hulme (1997) argues that assessment indicators for microfinance programmes must be precise and measurable, and the author classifies the assessment indicators into two

categories: economic indicators and social indicators. Economic indicators include the levels and patterns of income, expenditure, consumption, and assets. Social indicators such as individual control over resources, involvement in household and community decision-making, levels of participation in community activities and social networks have been extended into the socio-political arena in an attempt to assess whether microcredit can promote empowerment (Hulme, 2000, 1997).

The commonest methods used in IA include sample survey, rapid appraisal, participant-observation, case studies and participatory learning and action (PLA) (Hulme, 2000). Table 3.3 provides a summary of the IA methods including a description of the key features of each method. Each method has its own strengths and weaknesses. Hulme (2000) suggested that impact studies should adopt pluralistic approaches instead of a single method to avoid the weaknesses of individual methods.16

Table 3.3 Common impact assessment methods

Method Key Features

Sample Surveys Collect quantifiable data through questionnaires. Usually a random

sample and a matched control group are used to measure predetermined indicators before and after intervention.

Rapid Appraisal A range of tools and techniques developed originally as rapid rural

appraisal (RRA). It involves the use of focus groups, semi-structured interviews with key informants, case studies, participant observation and secondary sources.

Participation Observation

Extended residence in a program community by field researchers using qualitative techniques and mini-scale sample surveys.

Case Studies Detailed studies of a specific unit (a group, locality, organisation)

involving open-ended questioning and the preparation of ‘histories’.

PLA The preparation by the intended beneficiaries of a program of

timelines, impact flow charts, village and resource maps, well-being and wealth ranking, seasonal diagrams, problem ranking and institutional assessments through group processes assisted by a facilitator.

Source: Hulme (1997, p8)

One major obstacle to assess the impacts of microcredit programmes is the difficulty in addressing the ‘attribution’ or determining the ‘counterfactual’ (Islam, 2007; Baker,

16

For detailed comparison between the strengths and weaknesses of these methods, see Hulme (1997, p9).

2000; Hulme, 2000, 1997). Attributing specific effects (impacts) to specific causes is at the core of impact evaluation. In other words, how observed changes or impacts are attributed to microcredit or what would have happened in the absence of microcredit (Aghion and Morduch, 2005; Mosley, 1997). However, changes or impacts after a project intervention (like microcredit) may have been affected by other factors irrelevant to the particular project being evaluated, which makes the attribution of an observed change or impact to the project under evaluation difficult (Islam, 2007).

The attribution problem can be demonstrated through experiments in which the treatment is randomly allocated among a well-defined set of people. The random allocation process itself then creates comparable treatment and control groups that are statistically equivalent to one another given appropriate sample sizes. In theory, the control group automatically generated through this experimental design can serve as a perfect counterfactual in that it is assumed to be identical to the treatment group except for the difference in accepting treatment 17 (Baker, 2000). Thus the comparisons made between the treatment group and the control group established through an experimental process are considered to be an accurate estimate of the impact of the given project (or treatment) (Baker, 2000; Hulme, 2000).

However, experimental designs are thought to be unethical in social science due to the denial of benefits to otherwise eligible members of the population for the purposes of the study, and as a result, are relatively difficult to conduct (Baker, 2000; Khandker, 1998). Alternatively, impact evaluations of anti-poverty programmes such as microcredit programmes resort to non-experimental (or nonrandomised) designs to establish comparable control groups as similar as possible to treatment groups through econometric techniques. These techniques include matching method (propensity-scoring matching), difference-in-differences (double difference) method, and instrumental variables method (Baker, 2000). Table 3.4 provides explanations on

17

the different methods used in non-experimental impact assessment designs.

Tab le 3.4 Methods used to conduct quasi-experimental designs Method Description

Matching method In which one tries to construct an ideal control group that

matches the treatment group from a larger survey;

Propensity-scoring matching Most widely used matching, in which the control group is

matched to the treatment group on the basis of a set of observed characteristics or by using the “propensity score” (predicted probability of participation given observed characteristics); the closer the propensity score, the better the match;

Double difference or difference-in-differences method

In which one compares a treatment group and control group (first difference) before and after a programme. It is sometimes combined with the use of the matching method.

Instrumental variables (IVs) method

In which one identifies one or more variables that affect participation but not outcomes given participation and applies the IVs to predict programme participation, then sees how the outcome indicators vary with the predicted values

Source: Baker, 2000.

However, unlike experimental designs in which the selection of the treatment and control groups is random, non-experimental designs select the treatment and control groups after an intervention using non-random methods, which may give rise to a number of biases such as sample selection bias. Selection bias arises mostly from the unobserved or unmeasured characteristics, such as individual abilities, pre-existing conditions, and a subjective (often politically driven) process of selecting programme participants (Islam, 2007; Aghion and Morduch, 2005; Baker, 2000). These unobserved characteristics may bias the estimation of outcomes being investigated, including under or over estimations of actual programme impacts, negative impacts when actual impacts are positive (and vice versa), and statistically insignificant impacts when actual impacts are significant (and vice versa) (Aghion and Morduch, 2005; Baker, 2000). Baker (2000) argues that statistical techniques such as matching and using instrumental variables can possibly control for selection bias, but cannot fully remove it, leaving a major challenge for impact assessments.