Análisis Institucional

Results in this study noted three types of evaluations that emerged as key themes. Figure 1 captures the interrelationship between these themes as an evaluation framework for sustaining the impact of educational development projects. The illustration in Figure 1 starts with context evaluation prior to an educational reform project in order to design and plan the project linking with three layers (school, organisations and a central Ministry of Education) in an educational system. Context evaluation can inform both outcome and process evaluations as a basis for judgment for determining the success of the project, as well as assessing daily activities. Once the project is launched, the process evaluation for each activity at the three layers of capacity in the educational system begins—this is not well addressed in current project designs and implementation. In addition, periodical outcome evaluations re-examine the progress of activities assessed by process evaluation. Since the outcome and process evaluations aligned to the project funded by an international aid agency stops when the project is completed. Some of the practice of context and outcome evaluation, and much of process evaluation is expected to remain as an institutionalized evaluation system in the local educational system in the hope of the lasting impact of the project in a long-term period.

The conventional monitoring and evaluation for educational development projects do not appear to address sustainability of a project beyond its life time. As paradigm shifts occurred in international development as a consequence of aid ineffectiveness, evaluation practices for educational development is also required to change. Results from this study and current literature indicated that paradigms should shift from outcome-focused evaluation currently dominated by international donor agencies to process evaluation conducted largely by local participants but also supported by donor agencies. Process evaluation involves different layers of participants in the evaluation processes from the designing phase of an educational development project, so that they can not only acquire and improve evaluation skills, but also understand more about the projects. The practice needs more time to take hold in an educational system and is subject to personal views. However, process evaluation is expected to lead towards improving national evaluation capacity as a whole, being locally institutionalized. Therefore, this study concludes that the devised evaluation framework with process evaluation can be an approach for sustaining the impact of educational development in developing countries.

Special thanks to NCERD researchers for helping translations of Arabic data, logistics of accessing the participants and other research arrangements in Egypt. Further gratitude is expressed to the JICA Cairo office for granting permission to conduct this study.

References

Bamberger, M. (2000). The evaluation of international development programs: A view from the front. American Journal of Evaluation, 21(1), 95-102.

Burns, R. B. (2000). Introduction to research methods. London: Longman.

Canadian International Development Agency. (2000). RBM handbook on developing results chain. Quebec: CIDA.

Crawford, P., & Bryce, P. (2003). Project monitoring and evaluation: A method for enhancing the efficiency and effectiveness of aid projects implementation. International Journal of Project Management, 21, 363-373.

Creswell, J. W. (2008). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. (3rd edition). Columbus, OH: Merill Prentice Hall.

DAC Network on Development Evaluation. (2004). Evaluation systems in DAC members' agencies: A study based on DAC peer reviews. Paris: DAC.

DAC Network on Development Evaluation. (2008). Evaluating development cooperation. OECD DAC Network on Development Evaluation. Retrieved January 6, 2010, from http://www.oecd.org/dataoecd/3/56/41069878.pdf

Davidson, E. J. (2005). Evaluation methodology basics: The nuts and bolts of sound evaluation. Thousand Oaks, CA: Sage.

Development Assistance Committee. (2005). DAC network on development evaluation. Retrieved October 1, 2006, from

http://www.oecd.org/department/0,2688,en_2649_34435_1_1_1_1_1,00.html Development Assistance Committee. (2007). Glossary of key terms in evaluation and

results based management. Paris: OECD.

Development Assistance Committee. (2010a).Capacity development. Retrieved May 25, 2010, from

http://www.oecd.org/document/42/0,3343,en_2649_34565_42393642_1_1_1_1, 00.html

Development Assistance Committee. (2010b).The Bonn workshop consensus. Retrieved May 25, 2010, from http://www.oecd.org/dataoecd/34/31/42389348.pdf

Dykeman, M., MacIntosh, J., Seaman, P., & Davidson, P. (2003). Development of a program logic model to measure the process and outcomes of a nurse-managed community health clinic. Journal of Professional Nursing, 19(4), 197-203.

Earl, L., Katz, S., & Watson, N. (2003). Large-scale reform: Life cycles and implications for sustainability. Reading: CeBT.

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation: Alternative approaches and practical guidelines (3rd ed.). Boston: Pearson.

Fukuda-Parr, S., Lopes, C., & Malik, K. (2002). Capacity for development: New solutions to old problems. London: Earthscan.

Hashimoto, K., Pillay, H., & Hudson, P. (2008). Evaluating teacher education reform projects in developing countries: A case study of teacher education reform in Egypt, International Journal of Learning, 15(1), 123-131.

Hilderbrand, M. (2002). Overview: Meeting the capacity development challenges: Lessons for improving technical cooperation. In S. Browne (Ed.), Developing capacity through technical cooperation: Country experiences (pp. 15-36). London: Earthscan Publications.

Hong, H. D., & Boden, M. (2003). R & D programme evaluation-theory and practice. Burlington, VT: Ashgate.

House, E. R., & Howe, K. R. (2000). Deliberative democratic evaluation in practice. In D. L. Stufflebeam, G. F. Madaus & T. Kellaghan (Eds.), Evaluation models: Viewpoints on educational and human services evaluation (pp. 409-422). Boston: Kluwer Academic Publishers.

Independent Evaluation Group. (2007). Sourcebook for evaluating global and regional partnership programs: Indicative principles and standards. Washington, DC: The World Bank.

Ishida, Y. (2007). Evaluation on Education Improvement Projects form the Viewpoints of Contribution to Community Empowerment. Japan Evaluation Research, 7(1), 61-71.

Japan International Cooperation Agency. (2003). Egypt Arab Kyowakoku Syogakko Risuuka Kyoiku Kaizen Project Zissi Kyogi Hokokusyo (JICA preliminary study on project for improvement of science and mathematics education in primary schools in Egypt). Tokyo: JICA.

Japan International Cooperation Agency. (2004). JICA Jigyo no Yukosei to Zizokusei wo takamerutameni:Capacity Development Handbook (Improvement of efficacy and sustainability of JICA projects: Capacity development handbook ). Tokyo: JICA.

Japan International Cooperation Agency. (2005). Egypt Arab Kyouwakoku Shyogakkou Risuka Kyoiku Kaizen Project Tyhukan HyokaThyosa Hokokusyo

(The mid-term evaluation report on the project on improvement of science and mathematics education in primary schools in Egypt ). Tokyo: JICA.

Japan International Cooperation Agency. (2006a). Egypt Arab Kyouwakoku Shyogakkou Risuka Kyoiku Kaizen Project Syuryozi Hyoka Thyosa Hokokusyo (The final evaluation report on the project on improvement of science and mathematics education in primary schools in Egypt ). Cairo: JICA.

Japan International Cooperation Agency. (2006b). JICA's approach to capacity development. Retrieved August 2, 2009, from

http://www.jica.go.jp/english/about/policy/cd/index.html

Mabuchi, S., & Kuwajima, K. (2004). Tojyokoku no Capacity Development to Yuko na Enzyo (Capacity development in developing countries and effective approach). Journal of International Cooperation, 20(1), 64-72.

Mabuchi, S., & Yokozeki, Y. (2004). Genshyoku Kyouin Kenshyu Zitusshi Nouryoku no Teichyaku ni Mukete: JICA Risuuka Kyouryoku wo Capacity Development de Yomitoku (Toward establishing local capacity for in-service training: Unravelling capacity development through JICA’s assistance for science and mathematics education). Journal of International Cooperation, 20(2), 10-20. Matsuoka, S., Murakami, K., Aoyama, N., Takahashi, Y., & Tanaka, K (2008).

Capacity development and social capacity assessment, The Japanese Journal of Evaluation, 8(2), 3-24.

Minamoto, Y. (2007). Non-formal Kyoiku Enjyo ni okeru Sanka gata Hyoka Shyhou no Katuyo Rigai Kankeisya ga Hyoka Katei ni Hyoka Shutai to Kakawarukotono Igi (Utilizing participatory evaluation in non-formal educational cooperation projects: Relevance of stakeholders involvement in evaluation process), The Japanese Journal of Evaluation Studies, 7(1), 73-86.

Minamoto, Y., & Nagao, M. (2006). Process considerations in evaluating educational cooperation projects. Journal of International Cooperation in Education, 9(1), 89-105.

Minato, N (2004). Kaihatsuenjo no Hyoka to Enjyo Kikan no Doukou (Evaluation for Development Assistance and Trend in Aid Agencies) In Alkiyama, T & Kondo, M. (Eds.), Trend Series in Development Assistance 3, Development Approach and Changing Sector Issues (Kaihastu Enjyo Doukou Series 3, Kaihatsu Approach to Henyousuru Sector Kadai) (pp. 73-88). Tokyo: Foundation for Advanced Studies on International Development.

Miyoshi, K., & Stenning, N. (2008). Designing participatory evaluation for community capacity development: Theory-driven approach, The Japanese Journal of Evaluation Studies, 8(2), 39-53.

Nagao, M. (2003). Kyoiku Enjyo Hyoka no Genjyo to Kadai (Present concerns and issues in evaluation of educational cooperation). Journal of International Cooperation in Education, 6(1), 1-18.

Nagao, M. (2006). Challenging times for evaluation of international development assistance. Evaluation Journal of Australia, 6(2), 28-36.

Nutbeam, D., & Bauman, A. (Eds.). (2006). Evaluation in a nutshell. Sydney: Mc Graw Hill.

Patton, M. Q. (2004). The roots of utilization-focused evaluation. In M. C. Alkin & C. A. Christie (Eds.), Evaluation roots: Tracing theorists' views and influences (pp. 276-292). Thousand Oaks, CA: Sage.

Picciotto, R. (2003). International trends and development evaluation: The need for ideas. American Journal of Evaluation, 24(2), 227-234.

Preskill, H., Zuckerman,B., & Matterhews, B. (2003). An exploratory study of process use: Findings and implications for future research. American Journal of Evaluation, 24(4), 423-442.

Scriven, M. (1991). Evaluation thesaurus (4th ed.). Newbury Park, NJ: Sage.

Segone, M. (2006). The evolution of the evaluation function. New York: International Program Evaluation Network, United Nations Children's Fund.

Smith, H. (2005). Ownership and capacity: Do current donor approaches help or hinder the achievement of international and national targets for education? International Journal of Educational Development, 25, 445-455.

Stufflebeam, D. L. (2000a). Foundational models for 21st century program evaluation. In D. L. Stufflebeam, G. F. Madaus & T. Kellaghan (Eds.), Evaluation models: Viewpoints on educational and human services evaluation (2nd ed., pp. 33-84). Boston: Kluwer Academic Publishers.

Stufflebeam, D. L. (2000b). The CIPP model for evaluation. In D. L. Stufflebeam, G. F. Madaus & T. Kellaghan (Eds.), Evaluation models: Viewpoints on educational and human services evaluation (2nd ed., pp. 279-318). Norwell: Kluwer Academic Publisher.

Stufflebeam, D. L. (2004). The 21st-century CIPP model: Origins, development, and use. In M. C. Alkin & C. A. Christie (Eds.), An evaluation theory tree (pp. 245-266). Thousand Oaks, CA: Sage.

Stufflebeam, D. L., & Shinkfield, A. J. (2007). Evaluation theory, models & applications. San Francisco: Jossey-Bass.

Tanaka, H.,(2010) Fillipine deno NGO Kyoiku Zigyo Hyoka: Facilitator no Yakuwari to Stakeholder no Empowerment ( Participatory evaluation of NGO educational project: Facilitator roles and empowerment of stakeholders), The Japanese Journal of Evaluation Studies, 10(1), 75-93.

The Joint Committee on Standards for Educational Evaluation. (1981). Standards for evaluations of educational programs, projects, and materials. New York: McGraw-Hill.

United Nations Children's Fund. (2004). Changing lives of girls: Evaluation of the African girls' education initiative. New York: UNICEF.

United Nations Population Fund. (2006). UNFPA policy statement on results-based management. Retrieved March 6, 2009, from

http://www.unfpa.org/results/docs/policy.doc

World Bank. (2004). 2004 Annual report on operations evaluation. Washington, DC: World Bank.

Yin, R. K. (2003). Case study research: Design and methods (3rd ed. Vol. 5). Thousand Oaks, CA: Sage.

Figure 1: A Conceptual Evaluation Framework for Educational Development Projects

Before Project Life of Project After Project

Ongoing Process Evaluation Periodic Outcome Evaluation

Context Evaluation Local Institutionalized Evaluation System