In total four types of data were collected: mathematics performance tests, attitude questionnaire, observations and students’ lesson scripts12.
140
The performance tests and attitude questionnaires were given to the students together at the same time. First the students completed the attitude questionnaires then the mathematics tests. Students spend 10 minutes on the questionnaire and 35 minutes on the short test.
A timetable with all data-collection times was provided to each school. The questionnaires and the short tests were all administered and collected by the teachers with explanations from the researcher in terms of timing and how to operate the questionnaire.
The observations were conducted during the last two lessons of the project, so that the students had plenty of time to familiarise themselves with peer tutoring.
ICAT lesson scripts were collected by the teachers at the end of each lesson and were placed back in the folder provided to them.
5.5.1 Performance
Students’ mathematics performance was measured via three researcher-made instruments, reflecting both the students’ age and the mathematics topics covered during peer tutoring. The tests contained 3-4 questions for each of the six curriculum topics covered during the last 6 weeks of the year, for each of the schools.
5.5.2 Attitude
The questionnaire consisted of five sub-scales taken from various instruments:
The mathematics self-concept sub-scale was taken from Marsh’s ‘Academic Self- Description Questionnaire 1’ (ASDQ-1) (1990), with his consent.
141
Mathematics enjoyment, relating to people in mathematics classes, and choice of how to do mathematics in class were all taken and modified from Ryan and Deci’s (2012) ‘Intrinsic Motivation Instrument’ (IMI) (IMI website), with their consent.
The social self-concept scale was a researcher adapted instrument, however, inspired by Harter (1985), with her consent.
5.5.3 Social and academic behavioural observations
This study employed general school observation as well as pair observation. In order to measure the extent of peer tutoring implementation two classes were randomly chosen from each school. At the beginning of the observations the names for the tutee and tutor were taken for each desk for organisational purposes. Each pair was observed through 5 windows, specifically the researcher stopped five times a meter away behind each pair. The procedure was similar to the observations conducted by Topping, et al., (2011).
It must, however be made clear that there are many issues with observation methods in general. For example, one of the main issues with observations is that it is difficult to gain a representative sample of the participants or the process of learning in in general (Cohen, Manion, & Morrison, 2000). Consequently, a researcher could end up observing participants which belong to a background with certain characteristics, or observe only certain learning processes.
Moreover, the following have also been identified to be some of the limitations associated with observation techniques: a) influencing students’ behaviour, as they could perceive the observations as evaluations, b) not being able to know what goes on in students’ mental engagement, and c) limited resources and time, together with evaluating the broader context have implications for reliability and final interpretations (Cohen, et al., 2000).
142 5.5.4 Lesson materials
Each of the four parts in the ICAT framework reported above in the intervention section, were used to: a) conduct analysis at the tutee group level, specifically between the higher performing group and the lower performing group of tutees, and b) conduct implementation analysis at the school level.
5.6 Chapter conclusion
This chapter has reported information on the participants, schools’ characteristics, the intervention procedures such as group creation and characteristics, the design and data collection, exercises, training, and ethical issues, the ICAT framework. Also, the chapter briefly pointed out the instruments used for the intervention.
Three schools participated in the study, one school consisted of Year 6 students tutored by Year 8, Year 7 tutored by Year 9, and Year 8 tutored by Year 10. The schools were all reported as good schools by Ofsted, however, they differed from one another in terms of the percentage of students receiving free school meals, and the percentages of children in SEN and EFL categories.
The design for the intervention was the same for two out of three schools, with one of them deciding to implement ICAT to the entire school. The two schools with the same design, adopted a pre-post-test quasi-experimental design, and the remaining school a pre- post-test single group design. For the two quasi-experimental schools the groups prior to the intervention consisted of similar groups in terms of free school meals, however not in terms of previous performance, the better predictor.
143
The intervention was the same across the schools, one session of training and 6 sessions of ICAT. All teachers received the same amount of training in ICAT. Exercises in all schools were chosen by the teachers of each school to reflect their students’ ability levels.
All schools applied the same data collection techniques, the same instruments in terms of attitude questionnaires, collecting lesson materials, observation techniques, and similar performance test, differing only in the topics and levels.
144