Marco Conceptual

The literature on IMF operations is more extensive, but those studies directly associated with the Fund tend to be narrow in their approach, relying almost exclusively on econometric techniques. Important contributions are reviewed in Ul Haque and Khan (1998). The core narrative of this IMF review paper is one of ever more complex

techniques being applied to the same basic data to reveal progressively stronger impacts of IMF programmes. However, the discussion begins with a review of the work of Killick (1984 and 1995).

His two contributions stand out within the mainstream literature on the IMF operations, in that he relies on both quantitative and qualitative methods, and adopts a more overtly critical approach. Killick is also the most referenced independent source, and as the second study devotes specific attention to the Enhanced Structural Adjustment Facility (the IMF lending facility which would later become the PRGF) its content is particularly relevant to this thesis. Although the following discussion concentrates on Killick’s second volume the methods adopted in the two studies have much in common. Both offer a synthesis of multiple case study materials and statistical comparisons but the former does also employ cross-sectional regression analysis. The findings vary somewhat, with the initial study finding no real gain in the key target variables, whereas the latter shows clear but limited impacts. In contrast to other contributors, Killick employs a before-after counterfactual which compares post-policy outcomes against past performance. However, he also discusses a number of with-without comparisons made by other studies, and examines performance against planning targets within case materials. His work provides something of a menu approach to evidence gathering and appraisal, in that he draws on a number of techniques and analytical traditions.

The second study, Killick (1995), employs 17 detailed case studies, and given the presence of multiple arrangements, provides an appraisal of 48 IMF operations. Although a scrupulously neutral line is adopted in examining the IMF’s performance, it must be recognized that Killick accepts, without much scrutiny, the theoretical basis of the IMF’s monetary approach to the balance of payments, and thus he assumes a priori the potential effectiveness of stabilization policy. The key research objectives are an examination of impact performance and, within this, the character of IMF-government relationships and the role played by conditionality.

The findings as they relate to this thesis can be summarised under three headings. First, with regard to the impact of IMF conduct, Killick (1995) concludes that although controversy dogs individual stabilization packages, both supporters and critics tend to

overestimate the Fund’s power to enforce policy choices and to coerce national governments. He substantiates this by showing that financing operations are managed far less rigorously than the rhetoric (on both sides) suggests. Although the IMF has the option of cancelling a programme, the more common outcome is the issuance of waivers and/or post suspension renegotiation (often on terms favourable to the borrower). He also finds evidence of arbitrariness in performance assessment processes, and shows that certain countries have faced less stringent penalties for deviating from their agreed programmes.

Second, on outcomes, his findings bear some similarity to those of Mosley, Harrigan and Toye (1994). Gains were most evident for stabilization variables, whereas growth and inflation outcomes were generally weaker and in many cases not statistically significant.

These results are mildly encouraging for the Fund, but the magnitude of the stabilization gains was disappointing when compared against the targets. The analysis also provides some support for critical accounts of the level of disabsorption that followed IMF interventions, and that this was largely achieved through cuts in investment rather than consumption. By the same token, however, Killick (1995) finds that public expenditure reductions were generally achieved by cutting capital, as opposed to current expenditures, thus, he questions the claimed level of social retrenchment. On the supporting policy variables - exchange rates and domestic credit creation - the study finds the impacts are also far more muted than might be expected. On the longevity of the impacts, although evidence was found of some longer term benefits, there was no catalytic effect on future capital flows.

Third, in relation to the ESAF (i.e. the poorest) countries, Killick (1995) found the same pattern of outcomes, except in relation to growth, where the evidence of any gains was still weaker. Although, he offers no direct evidence on the impacts on poverty levels, the study references other sources, which found that the wages share of national income generally declines under IMF programmes, and from his own data, prices generally rise.

This is a very disappointing outcome for the Fund, which specifically tailored ESAF to meet the needs of the poorest borrowers, and it augured poorly for future PRGF performance.

It is worth underlining that Killick’s most striking finding, in both studies, is that both the IMF and its vocal critics over estimate the impact of its programmes. He argues that they rarely succeed in stabilizing vulnerable economies for any sustained period of time, and equally rarely do they impoverish entire populations. An important conclusion is his characterization of IMF conditionality as a “toothless tiger” and wholly inadequate for its claimed task (Killick, 1995, page 121). This is partly due to the nature of the relationship and conflicting incentives, but crucially also because the IMF’s practice has been poor – sometimes exercising too much leniency, but at other times applying it too strictly and in too doctrinaire a manner. He also argues geopolitical influences are important in shaping IMF decision making, and that a group of chronic borrowers emerged, and were in effect, engaged in an iterative game with the Fund. These points very much anticipate the debates that informed the development of PRSs some five years later.

The bulk of the other contributions within this branch of the literature are almost exclusively econometric in nature. It is also possible to identify a growing complexity in the approaches employed to control for initial conditions and other forms of unobserved heterogeneity, and to allow for selection biases. This prompted IMF authors Ul Haque and Khan (1998), who provide a review of the literature, to refer to the later studies as adopting a “Generalized Evaluation Estimator (GEE)”, which they define by the use of more advanced panel methods and instrumental variable (IV) techniques (Ul Haque &

Khan, 1998, page 10). While early contributions found mixed evidence, later and more technically sophisticated studies adduce clear beneficial impacts on stabilization variables and lagged growth effects. However, the magnitude of the growth improvements is weak, and this is disappointing given the progressively greater importance placed on growth within ESAF operations. Moreover, most of these contributions also found negative output impacts in the short term, with growth rebounding in later years as adjustment sets in (Ul Haque & Khan, 1998).

Problematically, these studies do not allow for variations in compliance (via slippage variables), or the local policy dynamics identifiable from case study evaluations. Thus they offer no real substantive account of the causal processes through which any gains are achieved. The veracity of their claims is also not helped by the relative weakness of the results for the key intermediate policy variables or the relatively poor overall explanatory

power they record. Troublingly, these contributions recognize few of the econometric challenges or the limitations of the techniques used.

In summary, the PBL literature both provides the policy context for the evaluation of the PRS Initiative undertaken within this thesis, and it serves as a methodological guide. It is also striking that the contemporary controversies over PRSs have much in common with the debates over past IFI lending operations. With regard to specific methods, several of the studies cited provide a technical resource for the examination of PRS performance.

The discussion also underscores the value of a mixed quantitative and qualitative evaluation, based on a sound counterfactual. However, it is also important to recognize the specifics of the PRS approach; the formal identification of poverty as opposed to growth as the primary goal of policymaking; and the changed relationship between the IFIs and recipient governments.