When conducting a national- or system-level diagnostic as defined in Chapter 1, a national-level stakeholder, such as a ministry of education (to use a generic term), may want to establish a nationwide baseline or to study the differences in reading ability among various groups—for example, second, fourth, and sixth grades—and determine average intergrade progression. Or, a national stakeholder might want to determine whether achievement differs between types of school systems (public vs. private) or curricular modalities (dual or monolingual programs).
In 2008 and 2009, Nicaragua and Senegal embarked on national-level diagnostic assessments of reading using the early grade reading assessment (EGRA) instrument. The purpose in both countries was to analyze—rigorously and in detail—the reading ability of children in the early grades and to examine the factors that may be responsible for those outcomes.
Although the two large-scale assessments discussed in this chapter had the same overall purpose, they differed not only by location but also by key parameters such as the implementation partnerships and language of testing. By comparing and contrasting them here, we aim to analyze what we know about the process of applying EGRA to the national reading diagnostic model.
The main focus of the chapter is on comparing the processes of planning,
implementation, and use of results; we examine specifically the choices that
were made at each stage. In both countries, the choices were made based on experiences in other countries, national preferences and needs, and client priorities. These choices had significant implications for both short-term efficiency and long-term impact of the assessments.
For RTI International, providing technical assistance to a country or a client to investigate the state of early grade reading is usually a short-term activity; it is limited in time and has a specific scope that begins with planning and
ends when we complete the data analysis and produce a final report. For the national stakeholders, however, the planning and implementation processes may actually be as important for education system reform as the results of the diagnostic, and attention to reading outcomes certainly does not end with the production of a final report.
In the cases of both Nicaragua and Senegal, RTI’s role extended beyond implementation of the diagnostic testing to targeted dissemination and communication of results. However, the activities differed greatly in the degree to which RTI and its local subcontractors engaged national counterparts. The relative level of ownership may account for contrasts in subsequent use of the results, both by ministry staff and civil society partners (i.e., voluntary as opposed to government-affiliated organizations), as discussed below. For example, the national diagnostic in Nicaragua was conducted with the full support and involvement of the Ministerio de Educación (Ministry of Education), including ministry staff who participated in the adaptation and training processes. After the results were analyzed, the ministry took immediate, positive steps to address the quality of instruction. By contrast, in Senegal, the national diagnostic was funded and implemented mostly independently of the Ministère de l’Enseignement Préscolaire, de l’Elémentaire, du Moyen Secondaire et des Langues Nationales (Ministry of Education), and the dissemination of results led to little discernable action by the ministry. Instead, civil society became engaged in the follow-up processes.
The chapter is organized in the following manner: First we give some background on the country context and the characteristics of the two national diagnostics, including brief results for each country. The second section explains how each country applied the national diagnostic model and what activities took place after the data collection and analysis were complete. Next, we review the similarities and differences between the two cases, and finally, we provide several key lessons that can be taken away from a comparative analysis of the two EGRA implementations.
Although we present summary results from both countries, the intent is not to compare scores, or to analyze comparatively the factors leading to those scores. For more detailed analysis of each country’s EGRA outcomes, please refer to the final technical reports for each country (CIASES & RTI International, 2009; Pouezevara et al., 2010).
Table 2.1 provides an overview of the key characteristics of each case, including the funding source, the driving policy question, and parameters of the assessment.
Table 2.1 Comparison of the two national EGRA diagnostics
Element Nicaragua Senegal
Funding source • United States Agency for • William and Flora Hewlett International Development Foundation
(USAID)
Purpose • To evaluate outcomes in schools • To examine the effect of that received USAID funding the classroom environment
(including language of instruction) on reading Target group • Grades 2–4 in public and private • Grade 3 (cours élémentaires 1,
schools or CE1) in French-language
public schools only
EGRA subtasks1 • Concepts about print • Letter identification: names and/ • Letter identification: names2 or sounds2
• Phonemic awareness • Nonword reading2 • Familiar word reading2 • Oral reading fluency2 with • Nonword reading2 reading comprehension • Oral reading fluency2 with
comprehension (subtask with two parts)
• Listening comprehension • Dictation
Other research • Teacher questionnaire • Teacher questionnaire instruments • Director questionnaire • Director questionnaire
• Classroom observation tool Sample size • National-level sample of 6,649 • National-level sample of 687
students students
Implementing • RTI International/Centro de • RTI International/FocusAfrica agency Investigación y Acción Educativa
Social (CIASES) with Ministry of Education
Communication • Policy dialogue workshop • Stakeholder workshop and dissemination • Teacher training workshops • Civil society communication activities • Video production training and workshop
• National campaign • Communication minigrants to civil society organizations • Mass media social-mobilization
campaign
1 As explained in Chapter 1, the subtasks chosen for inclusion in a particular version of the EGRA
instrument, and the names of the subtasks, may vary somewhat from one application to another depending on the purpose, language, country, timeline, funding resources, and other contextual factors. In all cases, however, the subtasks represent similar, typical categories of skills or abilities that individuals have to master in order to learn to read. For examples of the subtasks, see Appendix B.