METHODOLOGY TO EVALUATE
RESULTS OF PROJECTS FOR
INTEGRATION INTO THE
PRODUCTION CHAIN
EL ROL DEL BANCO EN PROYECTOS DE COMPETITIVIDAD: CLUSTERS
The Question
• How can Projects funded by the MIF for
Integration into the Production Chain be
successfully evaluated?
– How may effects of integration and
competitiveness be assessed together?
– What factors (local or sector) affect the success
of projects?
This Presentation
1. The theoretical context “in a nutshell”
2. The actual operating context and results of
our field analysis
3. The proposed methodology:
PART 1 - The Theoretical
Context
The Rational of an Integration Policy
Increase Exports and Improve Positioning in International Added Value Chain
Improvement of Production System Increase in
Collective Efficiency
Increase Profitability and Investments
PART 2 - The Operating Context
The Profile of “Cluster” Projects or IPs
• have relatively restricted financial resources
(<=1 million);
• utilize a limited number of suitable tools (training,
business services, institution building);
• take about 3 years to implement (the full cycle is
longer);
• are executed by private bodies;
Hence, the evaluation should…. (1)
•
focus on only a few types of intervention;
• be
simple and low cost;
• assume
limited and uncertain
direct effects on
competitiveness;
• expect an increase in
business cooperation
as a
condition to improve competitiveness;
Main Results of Field Analysis
• The clusters are composed of micro or small businesses, operate in the same production segment, show limited product differentiation, and generally have independent outlets to the market
• businesses are seldom integrated into a value chain and
do not exploit the advantages of proximity
• lack of public policies to modernize processes and
products (access to credit, grants, etc.) reduces the impact of projects
Hence, the evaluation should….(2)
• Support the
identification of clusters
•
Improve planning
and consequently increase the
degree of success
• Help the projects to
exploit all the potential
effects
(economic, social, institutional, etc.)
• Standardise its implementation and results in order
to
facilitate benchmarking and “learning by
The actual procedures:Phases, Players, Problems
MIF, potential EA CO, Reg.Dept. REG.Dept. (Team leader) Board, MIF, IDB Washington CO, EA CO, EA
CO, EA, OVE,
MAIN PLAYERS
9No clear and shared responsibilities
for results among planners and managers
9Standards for approval as such that projects are often far from reality
(too ambitious objectives)
9Lack of in depth preparatory
studies, evaluability conditions, and baseline for the expected results
9Slow start-up and changes in the foreseen conditions
9Monitoring of the activities and not of the results (formal control)
9No clear monitoring responsibilities
Problem identification Project definition Project definition Approval & contract Start up & management Controls
Hence, the evaluation should….(3)
•
Reinforce its role
in the process
• Be
based on clear responsibility
in each
phase
•
Favour the relationships
among the many
players involved
PART 3 - The Proposed
Methodology
The main effects of the cluster projects and the
“key” variables
Effects on the group of
treated enterprises (integration, size & typology) Effects on the
Variables Describing the Changes in
Competitiveness at Company Level
• Level of turn-over • No. of clients
• No. of products designed and sold jointly (i.e. fashion system) • Amount of sales outside the area
• Amount of exports
• Acquisition of new techniques and new machinery • New products, new packaging
• Creation of trade-marks
• No. of restructuring plans (reorganization, new skills, etc..) • No. of employers in continuous education
• Product/employer total or per production line
Variables Describing Changes in Cluster Size
• No. of firms located in the cluster area
• No. of production lines installed in participating firms • Amount of new investments and jobs created
• Types and value of services inherent to production • Level of disbursement for external services
• Rate of birth of firms • No. of new firms
Variables of Networking
• No. of inter-firm contracts
• No. of agreements for common treatment of: supply, exports, marketing, etc.
• No. of firms participating in projects
• No. of relationships between SMEs and leading firms • Percentage of co-financing from local authorities
• Quantity of joint sales
• No. of agreements with large scale international traders and wholesale traders
Choosing the right Indicators: the Hierarchy of
Objectives
Training
Business services
Introduce new skills and reinforce
professional attitude
Promote product and process innovation in SMEs
Develop human resources
Generate innovative ability in the area
The Different Types of Indicators for an
Integrated Approach
• Activities
Æ
financial and physical
indicators
• Immediate objectives
Æ
output indicators
• Intermediate objectives
Æ
result indicators
10 final points… in the identification phases
1. Targetting
a. existing or new cluster,
b. micro or small-medium enterprises, c. local market or national/export market
2. Defining right objectives
1. Relevance, in financial and strategic terms (How the intervention can influence the local needs?)
10 final points… in the planning phase
3. Including Evaluability in the LF
a. What (expected results on firms, cluster and local environment at micro level)
b. Who (responsibility for monitoring and evaluation) c. How (set of–financial, output, results and impact –
indicators focused on different aspects of the cluster, tested and shared with the EA)
d. Produce a baseline to compare with the results during the implementation
10 final points… in the start-up phase
5. Updating the evaluability
(adaptation to changes in the plan, verification of the indicators and the methods for gathering information, check of the baseline andeventual re-calculation on the final clients)
6. Including indicators of results in the reports
(extending the formal control of activities to the outputs and results of them)
10 final points… in the implementing phase
8. Reinforcing the role of the evaluation
1. producing a comparison of results with the initial baseline in the intermediate evaluation
2. verifing the functioning of the monitoring system
3. introducing new forms of involvement and diffusion of results – workshops of different projects, specific meetings with the stakeholders (EA,CO, final clients)