The Evaluation will be carried out in accordance with the Danida Evaluation Guidelines (January 2012) and the OECD/DAC Evaluation Quality Standards (2010). The
Evaluation must be based on a sound methodology. The Evaluation design must be methodologically rigorous and credible when judging both the internal and external validity of the results.
Annex A: Terms of Reference
Bidders are free to propose the most appropriate designs for responding to the evaluation questions indicated above.
This section provides some initial thinking on the approach and methodology which will need to be further developed by bidders in their technical proposals and, following contract award, in the inception report through concrete and practical solutions. The Evaluation is expected to proceed in three phases:
• Inception phase – in which key stakeholders will be identified, initial data collection will take place, the methodology and evaluation matrix will be refined (based on the outline included in the proposal), working hypotheses developed, and a detailed process action plan presented. This phase will include an initial stakeholder workshop and dialogue with key stakeholders in Copenhagen.
The IMSC secretariat will ensure that all relevant documentation is made available to the Evaluation Team. The inception phase will result in an inception report that will be presented in draft and final versions.
• Implementation phase – during which the way forward outlined in the inception report will be pursued. This will mainly involve data collection and its analysis. The Evaluation Team will undertake a field mission to the Horn of Africa Region (Kenya and Ethiopia) with the specific focus on the Horn of Africa Programme. The Afghanistan/Pakistan programme, will be covered by remote consultation with relevant stakeholders in Kabul and Islamabad, as well as relevant stakeholders in Denmark.
• Reporting phase – during which the evaluation team will develop its findings and present them in a draft and final evaluation report. Early in this phase, the team will facilitate a final stakeholder workshop at which preliminary findings will be discussed. As outlined below, this workshop will be held back-to-back with a separate workshop on Theory of Change for staff engaged with the management, design, programming and implementation as an input into the programming of the next phases.
• Final open seminar for stakeholders in Copenhagen where findings and conclu- sions are presented.
Tenderers are expected to expand on the above outline in their proposals.
The evaluation will primarily utilise qualitative methods drawing from the following: • Desk study of relevant programme documentation, including all programme
documents, reporting etc. as well as project documentation for strategic inter- ventions funded outside the programmes
• Stakeholder consultation in Denmark and internationally drawing from the evaluation questions as set out in the agreed evaluation matrix.
• Consultation of secondary sources, including evaluations/reviews, guidance/good practice and other learning sources related to stabilisation/fragile and conflict affected states.
Annex A: Terms of Reference
• Analysis of data collected. This will be in accordance with the DAC evaluation criteria and the evaluation questions outlined above and in the evaluation matrix (a first draft of which is to be presented in the tender). The analysis will also take into account theory of change issues (see below).
• Development of key findings arising from the analysis.
• Presentation of the evaluation findings and recommendations.
In order to support the Evaluation, the MFA EVAL with the support from the Inter- ministerial steering committee will provide the Evaluation Team with relevant qualitative
and quantitative data, including a mapping of interventions supported by the Fund,
budgets and disbursements etc. the Evaluation Team will incorporate this data into the evaluation process and reflect an analysis of it in the evaluation findings.
The quantitative analysis will include:
• A brief mapping and typology of interventions supported (to be proposed by the Evaluation team in the Inception Report, see below).
• An overview of budgets and disbursements according to a typology of interven- tions. This should be divided according to funding source (i.e. ODA/non-ODA) and modality (i.e. bilateral, joint, trust find etc.).
• Findings relating to the approach taken by Fund, disbursement rates etc.
Theory of Change has not been specifically addressed in the Fund so far. Recent reviews
have noted that this is not adequate and have suggested that a more explicit approach to Theory of Change needs to be integrated into the early phases of programme design so that there is a closer relationship between intended stabilisation results and the means available to achieve them. The Evaluation Team should, therefore, assess the interven- tions currently funded from this perspective and consider the steps that could be taken to utilise a practical and realistic approach to this in the future.
In addition to the Evaluation Team’s reflections on this in its reporting, the MFA would like the team to participate in a separate workshop on Theory of Change that will be held back-to-back with a stakeholder workshop towards the end of the Evaluation. The MFA will make the arrangements for this workshop, including contracting an expert facilitator. The audience for the workshop is expected to include officials from the MFA, MoD/ Danish Defence Command, and possibly the Prime Minister’s office and Ministry of Justice as well as a limited number of other practitioners. Tenderers should allow one day in their proposals for participation in this event.
To engage other key stake holders primarily in Denmark it is foreseen that the contract with the evaluators will encompass two stakeholder workshops in Copenhagen: One at the start of the Evaluation to get inputs and views and one at the end of the evaluation to discuss findings and conclusions. The financing of the workshops will be managed outside the evaluation budget.
Annex A: Terms of Reference