Cómo aprende la mayoría de las personas
1. La clase magistral
This section assesses the extent to which CCAFS is enhancing its effectiveness and impact through theory based implementation of research and integration of the program components both at the ToC level and operationally. The Evaluation team also analyzed the role that CCAFS gives to advocacy for promoting its concepts and enhancing its influence and impact. Assessment of effectiveness and achievement to date is provided at the FP and RP sections that follow.
Evolving theories of change
Subsections under 3.1 above set out the Evaluation team’s assessment of the ways that the Program is addressing the need for research on the ways that climate change is and will affect agriculture and food security, and how CCAFS is working toward CGIAR defined IDOs. These strategies combined with the FP and RP impact pathways are relevant to the overall and more detailed theory based processes of change that the Program seeks to inform, influence and bring about.
At different junctures CCAFS has made explicit the changes its research contributes to and how it expects to achieve impact. These ‘theories of change’ have evolved over the course of the Program. The Program Plan 2011 was followed by Business Plans for 2012, 2013 and 2014. These were superseded by the ToC in the 2015 Extension Proposal 2015-2016, which relates upwards to the CGIAR SRF 2015-2025. Further revision to the CCAFS ToC has taken place for the Phase II proposal.
While recognizing that the CCAFS ToC is an evolving target, the Evaluation team made a formative assessment of the ToC and impact pathways that CCAFS worked toward in the period immediately before and during the extension period. The processes used to develop regional impact pathways, and to meld a project portfolio under the FPs and RPs, which take forward and test the ToC, were assessed as part of the key evaluation question on the robustness of the MEL system. The findings are relevant to the proposed ToC for the second phase.
46
iea.cgiar.org The current CCAFS ToC is set out in the Extension Proposal40. It correctly recognizes the dual context of gathering momentum at the global41 and national levels toward climate action — largely but not entirely mitigation focused — and the fragmented, often ad hoc and short-term nature of local responses to emerging risks and opportunities resulting from increases in global warming and climate variability. In the Extension Proposal, CCAFS envisaged three broad ways to engage in these change processes by: generating evidence from action research; effecting policy and institutional change to support CSA, and; rolling out CSA. Care has to be taken where these processes by necessity run in parallel, that the evidence generated to effect policy engagement is couched in the necessary caveats of uncertainty.
The changes at the high outcome level CCAFS projects to achieve under the current ToC are largely aspirational in nature. These outcome targets to be achieved do not have great definition or differentiation and they are not well linked to the lower level milestones reported annually. Therefore assessment of the expected scale of change impacting end-users and the environment is difficult. Ambition levels vary across the Program. CCAFS staff have commented that, for example, the FP1 targets of farmers adopting CSA (not yet an outcome) could be achieved in one of two states of India alone, but this would miss the point of CSA being of benefit to smallholder farmers facing climate challenges in all continents. The contributions of FP1 and FP4 (the more cautious level of targets withstanding) to the accomplishment of the envisioned change appear more concrete and achievable than those of FP2 and FP3, recognising that there are few if any scientific ways to assess changes in adaptive capacity other than through longer-term impact evaluation with normalization for climate change. It is anticipated that the experience of phase 1 will allow the Program to develop a more realistic, more context specific (disaggregated to countries), and better specified ToC for phase 2. Each FP contributes to the Program ToC in ways reflected in the FP impact pathways toward outcomes. A more granular assessment of the work of FPs toward Program outcomes is presented in section 3.4.
Effectiveness of climate-smart agriculture
Recognizing that CCAFS’ shift to use the CSA framework has been fairly recent, good progress has been made in some aspects of the way CSA is being operationalized across the Program. As discussed in section 3.3.3, the CSA framework has been a useful tool to achieve better integration across projects and help drive change across centres participating in CCAFS. In this way the CSA framing has helped improve coherence of the individual FPs, and to some degree across the Program as a whole. Work has also started on developing criteria to help answer the question of what constitute CSA practices (e.g. the work on ‘certification’ in FP1 in SA). In some cases, CSA has also been a useful communication tool with boundary partners, although as discussed in 3.3.4, this is not without risk.
Despite these achievements, the Evaluation team’s view is that scientifically substantiating the CSA concept needs further work. There are several areas of weakness discussed below that require significant attention as CCAFS moves into the next phase.
40 CCAFS Extension Proposal p. 2 and summarized in Fig. 1 on the same document. See:
https://ccafs.cgiar.org/publications/ccafs-extension-proposal-concept-note-2015-2016#.VnajQL8elA4 41 as exemplified at the UNFCCC COP21 in Paris December 2015.
47
iea.cgiar.org The CSV concept may have been popularized by CCAFs, but it still lacks good definition. CSVs imply that the resident communities follow agricultural practices that are resilient to impacts of climate and effectively manage such risks to secure their nutritional and food requirements. Knowledge on how to achieve such villages through CSA practices would be instrumental for replication in other climate vulnerable localities and would constitute a unique IPG. CCAFS does not yet fully capitalize these opportunities.
CSVs are not now unique to CCAFS, and stakeholders from multi-lateral agencies, to donors and governments are touting them. CCAFS, being one of the pioneers of the concept and with the requisite science to support it, is in a good position to take the concept forward into practical implementation for others to follow. The Evaluation team suggests that more strategic partnerships outside the CGIAR System at this level will greatly help identify and tailor CSVs appropriate for each unique case42. The new partnerships with NGOs in Nepal and Colombia show promise.
The CCAFS framing of CSA at the time of this Evaluation, and its operationalization through CSVs did not recognize sufficiently and explicitly that increasingly, non-agriculture related livelihood strategies (off-farm, micro-enterprises, temporal or permanent migration) are valid and important alternative adaptation options. There has been failure to recognize the diversity of livelihoods of rural households—wider than just cropping43 and livestock production—that include migration and the option of exiting agriculture altogether as climate adaptation strategies. The household level trade- offs involved in opting for some of these pathways impinge on the attractiveness and feasibility of the CSA practices being developed and promoted by FP1, and need to be taken more into account by CCAFS researchers to magnify effectiveness (see section 3.4.1). Phase II is an opportunity to address these issues.
In addition to the need to ground the CSA concept in a livelihoods context and sharpen the way it targets the most vulnerable of rural households, in some regions the ‘village’ focus is also not appropriate and the CSV approach needs to be more watershed or landscape oriented – this was the case in SEA and LAM in particular. In general, the Evaluation team questions how representative CSVs are of their agro-ecological hinterland. In any assessment of impact, adjustment is needed for the residual effects of other interventions such as those from previous CGIAR programs at these sites. Indeed, questions can be raised about how well the CSV concept fits into and contributes to national agricultural development, other than being a special case of high-investment interventions. As a result, in SEA and LAM the CSV concept is being broadened to encompass climate smart landscapes and watersheds. The Evaluation team regards this as both pragmatic and positive.
A further challenge, noted in interviews with the SEA and LAM RPs, is that while the concepts of CSA are generally accepted, translation of the terms ‘smart’ and ‘climate’ is not always straightforward (e.g. in Vietnam), raising the question how effective the concept is in other cultural settings.
42 “Success factors include participative and locally driven vulnerability assessments and tailoring of adaptation technologies to local contexts, mapping local institutions and working in partnership across institutions.” Wright et a. 2014, Farmers, food and climate change: ensuring community-based adaptation is mainstreamed into agricultural programs, Climate and Development, 6:4, 318-328
48
iea.cgiar.org Irrespective of the above problems of a too narrow framing of CSA, achieving coherence across the three dimensions of production, adaptation and mitigation is also challenging but the rewards available from synergies make it worthwhile to pursue. Reasons for this are that there is very little support in either national plans or the UNFCCC negotiations for a cross-sectoral approaches; tension remains despite some progress as discussed in the introductory section 1.3.2. Mitigation and adaptation remain separate within UNFCCC negotiations; the IPCC maintains three different working groups and many climate funds will support either adaptation or mitigation but not both. Policy design for agricultural development and much of the research methodologies that inform policy-making likewise tend to consider only a single entry point.44
The Evaluation team identified positive examples of convergence, and also research activities that address only one, or at most two, of the three tenets of CSA. These are candidates for convergence with research addressing the reciprocal areas of CSA. The Evaluation team suggests that research activities that look at synergies and trade-offs be strengthened in the future. Further, much CCAFS research in support of NAMAs, INDCs or adaptation plans45 remains single sector in its approach, without much consideration of the impacts on other components of holistic climate responses. The Mitigation Options tool, the Ex ante Carbon Balance tool46 and the Climate Analogues tool aim either at mitigation or adaptation and their relation to food security; thus greater integration is needed. Achieving FPs convergence needs changes in both defining research topics (i.e. research questions that explore convergence) and methodology (i.e. how to do research that enhances convergence). It also requires behavioural change by different stakeholders including researchers, policy-makers, farmers and actors involved in the value chain.
Notwithstanding such difficulties, the Evaluation team notes that CCAFS has made some progress toward understanding synergies between adaptation and mitigation47 as well as on links between adaptation and food security,48 and between mitigation and food security.49 Stronger convergence is apparent in projects in SA, LAM and EA, while in SEA and WA FP1 is working toward coherence. In WA, CCAFS has achieved some convergence in CSVs, thanks partially to the science-policy platform led by the Ministry of Agriculture in Senegal, but unfortunately low-emissions agriculture is not
44 Ecker and Breisinger, 2012. The Food Security System. A new conceptual framework. International Food Policy Research Institute (IFPRI), Development Strategy and Governance Division; Ericksen et al. 2009. Food security and global environmental change: emerging challenges. Environmental Science & Policy 12, 373–377. Galaz et al., 2012. “Planetary boundaries” — exploring the challenges for global environmental governance. Current Opinion in Environmental Sustainability 4, 80–87.
45 National adaptation plan is a UNFCCC instrument that allows identifying medium- and long-term adaptation needs and developing and implementing strategies and programs to address those needs. See http://unfccc.int/adaptation/workstreams/national_adaptation_plans/items/6057.php
46 https://ccafs.cgiar.org/ex-ante-carbon-balance-tool#.VcTIbi8d79o
47 Ogle et al. 2014. Reducing greenhouse gas emissions and adapting agricultural management for climate change in developing countries: providing the basis for action. Global Change Biology 20, 1–6.
48 Vervoort et al. 2014. Challenges to scenario-guided adaptive action on food security under climate change. Global Environmental Change 28, 383–394; Wright et al., 2014. Farmers, food and climate change: ensuring community-based adaptation is mainstreamed into agricultural programs. Climate and Development 6, 318– 328.
49 Powlson et al. 2014. Limited potential of no-till agriculture for climate change mitigation. Nature Climate Change 4, 678–683; Valin et al. 2013. Agricultural productivity and greenhouse gas emissions: trade-offs or synergies between mitigation and food security? Environmental Research Letters 8, 035019.
49
iea.cgiar.org included consistently in the region. Converging approaches include the study of mitigation and adaptation synergies in coffee production50 (an important crop in Colombia, Peru and Central America), and developing regional scenarios for cross-scale research.51
On balance, the Evaluation team sees CCAFS moving in the right direction here, but in order to fully capitalize on its comparative advantage, CCAFS needs to place additional effort into approaches that more rigorously and systematically assess and provide ways to enable trade-offs to be identified and tested between food security, adaptation and mitigation prior to injecting findings into policy debates.
Integration
This section draws on the assessment of interview responses to KEQ 1. This KEQ cuts across the evaluation criteria of QoS and effectiveness and provides an additional lens through which Programs effectiveness can be evaluated. Integration is important if CCAFS is to produce high quality science and bring about tangible outcomes and impacts (see also section 2.2.1). In response to the CGIAR reform agenda, CCAFS has made significant progress in operationalizing integration to a greater degree.
The Program seeks integration at two levels. It brings together and builds on relevant component science outputs from other CRPs and outside of CGIAR, which requires CCAFS to undertake interdisciplinary research; CSA as an integrating framework facilitates this. CCAFS also seeks to integrate science outputs into decision making by next-users to affect change at scale. The Evaluation team believes that this level of integration requires greater emphasis in Phase II on a transdisciplinary approach, by which we mean the co-development of science-based and non science-based52 knowledge involving science and non-science actors. This is distinct from interdisciplinary research, which only integrates different science disciplines while remaining essentially science driven.
In assessing integration, the Evaluation team generally considered three levels: integration of component science in projects and activities; syntheses that take place in the FPs and some of the larger projects; and, integration with boundary partners to achieve outcomes and impact, mostly in the RPs. This is diagrammed with reference to SA in Figure 3-1.
Assessing project level integration, the Evaluation team draws a distinction between projects led by CGIAR centres (and usually funded bilaterally or through W3) and those commissioned by CCAFS. The former depend more on project leaders than on CCAFS core team members, and because CGIAR centres had a strong hand in which legacy projects were initially included in CCAFS, projects tended to be smaller, more narrowly focused and less connected. Projects that had a longer history pre- CCAFS, those closer to dissemination that drew on a wider range of boundary partners, and those where donors had a stronger influence on project scope, showed greater integration across a spectrum of issues and partners. Overall, integration and cross-centre and cross-CRP linkages were insufficient in Phase I. Integration was relegated to a ‘meta’ level, through synthesis of individual
50 Rahn et al. 2013. Climate change adaptation, mitigation and livelihood benefits in coffee production: where are the synergies? Mitigation and Adaptation Strategies for Global Change 19, 1119–1137.
51 Thornton et al. 2012. The role of regional scenarios in CCAFS cross-scale research, planning and action toward improved food security, environments and livelihoods. Internal note for CCAFS PMC.
52 Non science-based knowledge is derived from experiential learning and other forms of inquiry that do use scientific methods for knowledge creation.
50
iea.cgiar.org project-based work at the theme level. This changed in 2014, when several changes were implemented, in particular the introduction of the CCAFS ToC and impact pathways, which forced stronger alignment and integration of projects towards impact pathways and a consolidation from ~300 to ~90 projects. Large projects and clusters of projects are better linked in the extension phase and more integrative, particularly within regions.
Figure 3-6: Analytical framework for assessment of integration
Source: Evaluation team.
In general, the Evaluation team notes that integration in projects depends upon the project leader management style and this can be strongly influenced by the culture prevailing in the CGIAR centre where the leader is based. Some of the team leaders of cases assessed during the Evaluation do not have the necessary skill required for good integration. As a result, linkages and integration were both more variable and much less coherent in Phase I (2011-2014) than in the current portfolio.
Aside from project management style, the Evaluation team noted three other constraints to integration at this level. Firstly, while CSA and CSV have helped to drive integration to some extent, the Program needs to broaden its roll out of the CSA framework (see critique of narrow CSA framing in 3.3.2, and 3.5 for region-specific assessment). Secondly, the input-output relationships between projects need to be more explicit (see section 2.1 for assessment of the P&R platform as a management tool). Finally, the CGIAR operating environment—notably uncertainties surrounding budget amounts, availability and flexibility—results in slippage and loss of key partners, with knock on effects on projects that come next in a sequence, and therefore on overall effectiveness of delivery through the portfolio.
In conclusion, the Evaluation team considers that measured against the integration potential inherent within the CCAFS research portfolios, and comparing its practice against some other international organizations, more integration and linking is required.
51
iea.cgiar.org
Advocacy
To a large extent contributions to achieving the changes envisioned in the Program ToC through the FPs (particularly FP1 and FP4) requires advocacy (communication of research findings that promote one or other action to be taken53) of CSA technologies to next- and end-users, and the promotion of and advocacy for investments in climate smart policies and institutions. In part the need for advocacy is due to the perceived urgency in the need to move agricultural production toward lower carbon emissions to contribute to the UNFCCC ambition of global warming below 1.5oC, while noting the parity given to adaptation in the Paris Agreement. But there is also the need for advocacy for designing and implementing climate adaptation to future climate risks where the risks can only be explained in probabilistic terms. Here the Program faces ‘implicit discount rates’ in the ways that policy-makers and others offset costs of climate effects to the future. Advocacy is also necessary because the definition of what are CSA and institutional investments are neither based upon unequivocal science nor agreed among stakeholders. So the Program finds itself having to advocate what is believed to be