The collection of data for this study was carried out in two rounds. The first round data collection involved the collection of baseline data before the quasi-experimental intervention period on the use of quality formative assessment. The baseline data included determining the students’ perceptions on the practices of quality formative assessment and finding out whether this (if present) enhanced their self- regulation on learning and assessment skills. Besides, the baseline data included the pretest achievement scores of the students from a chapter of the course. During the second round data collection, data were collected by means of classroom observations, questionnaire administration, interviews, focus group discussions and a post-test on learning achievement.
As discussed in the earlier sections of this chapter, in terms of research paradigms, this study followed the pragmatist paradigm. The researcher conducted the study with a mixed-methods approach consisting of a main quantitative survey sequentially followed by qualitative data collection and analysis. The study was conducted at three of the six universities located in the western part of the Ethiopia. For each of the three universities, students of two intact classes (entire class of students) who were taking the “General Psychology” course particpated as intervention groups. Students from many academic programmes at universities enroll for the above mentioned course. The researcher chose this course because access to students of more than one entire class was possible, which was helpful in
105
quasi-experimental studies. Other students who were taught by the same educator of a given intact class also took part in the study as comparison groups and supplied quantitative data for the dependent variables of the study, which were achievement test scores and perceptions on the self-regulation of learning and assessment. The reason for the inclusion of students from an entire class is the principle of quasi-experimentation, which allows for convenient sampling through the use of naturally formed groups. The two subsections below briefly present the procedures of data collection for the quantitative and the qualitative phases of the study respectively.
3.12.1 Data collection procedures for the quantitative phase of the study
The quantitative phase on this study took the form of a descriptive survey and relied on data collected using questionnaires and achievement tests. According to Gay and Airasian (2003: 25), a descriptive survey is an appropriate method of studying the perceptions of a large number of respondents regarding a certain phenomenon because it describes the way things are. In terms of specific methods, the study applied a quasi-experimentation approach for the quantitative phase of the study. To collect data on the students’ perceptions on the use of quality formative assessment, a self-reported questionnaire with alternative responses to a four-point Likert scale ranging from “always not true (=1)” to “always true (=4)” was used. The content of the self-report questionnaire completed by the students included the enhancement of self-regulated learning and assessment (13 items), the practice and use of quality formative feedback on the course (17 items), the practice on self-assessment (15 items), and the practice on peer assessment (12 items). All questionnaire items were prepared by making reference to the literature reviewed in the second chapter of the study. Achievement test score data were collected using tests prepared by the researcher in consultation with the educators of the course (form A & B, see annexture 6 & 7). The data collection entailed using the self-report questionnaire (for the perception of self-regulating learning) and the achievement test (post-test) was repeated after the instructional intervention in which quality formative assessment was integrated and used on the lessons of the course.
3.12.2 Data collection procedures for the qualitative phase of the study
The research questions for the qualitative phase of the study were carefully set to determine the perceptions of the research participants on the use of quality formative assessment in the instructional process. Hence, the researcher executed the qualitative data collection by preparing questions that elicited the relevant information to answer the research questions. The qualitative data for this study
106
were collected by means of focus group discussions conducted with few students from the intervention groups, interviews with the educators who taught the course and classroom observations.
To supplement the results obtained in the quantitative data collection and analysis phase, the researcher carried out key informant interviews with the educators who participated in the teaching of the course by the integration of quality formative assessment on lessons. The interview data were captured by taking written notes of the main ideas raised by each interviewee. The purpose of the interview was to answer research sub question 4 and to examine:
The educators’ personal perceptions on the advantages of quality formative assessment on the improvement on the students’ learning;
The experiences and difficulties they faced while applying quality formative assessment in the lessons of a course;
The reactions of the students while quality formative assessment was used on the lessons.
In addition, the researcher faciltated six focus group discussions (ranging between six and ten students per FGD) with the students who were in the intervention group. The objective of the FGDs was to examine perceptions and experiences of the students towards the practice and role of formative assessment in advancing the self-regulation of learning and improvement. The researcher took written notes on the key ideas raised by the focus group discussants and in case of unclear ideas the discussants were asked for clarification. The researcher limited the number of FGDs to six because after the fifth one, data saturation was noted and new idea was hardly emerging from the focus group discussants next.
Furthermore, during the implementation of the instructional intervention that integrated the use of quality formative assessment on lessons, the researcher sat in on the lectures and acted as a complete observer on the activities of the educators and the students when using quality formative assessment in lessons. Each educator was observed five times and structured notes were taken on the activities of the educator and the students, the type and frequency of quality formative assessment used and on the activities of the students during assessment. More specifically, the observations focused on the actions of the educators and the students at the start of, during, and at the end of a given lesson, the classroom interactions as well as the type of formative assessment used on the lessons observed.
107
The researcher used the data collected from the above-mentioned participants in the study (educators and students) to find out the ways they perceived the use of quality formative assessment’s contributions to learning improvement and enhancement of self-regulated learning. Moreover, the researcher asked the particpants to reflect on their perceptions of the major impeding factors and the difficulties to implement quality formative assessment while teaching the course. To promote the success of this intervention, the researcher attempted to secure the trust and cooperation of the study participants to obtain the required data, and set out to create good rapport that could enhance the reliability and validity of the study results. This was made possible by the discussions carried out with the research participants before scheduling the actual data collection time. Furthermore, the researcher showed formal letters, which granted him the permission to conduct research at the universities (see letters on Appendices B, C, & D).