3.6.1 Testing
Tests can be used for evaluation purposes and are commonly used in quantitative research as “a powerful method of data collection” (Cohen et al, 2011, p. 476). Standardized tests are normally developed by psychometricians and according to Hidden Curriculum (2014) consist of a common bank of questions in which all participants are exposed to and is subsequently scored in a consistent manner, enabling the comparison of relative performance of individual or group participants. Non parametric tests are designed for a specific population and are valuable to teachers because of their ability to provide information from designated subjects (Cohen et al., 2011). Likewise, one of the main advantages for using norm-references tests (NRT) is to classify students, as well as highlight academic achievement between and among each other (Cox & Vargas, 1966).
3.6.2 Configuration of tests
The pre-tests were conducted over a one-week time period prior to the implementation of the reading unit implemented within the research framework. The student participants were administered the test individually under the guidance of the researcher, in a comfortable, familiar semi isolated classroom environment in order to promote ‘empowerment’ to the participants by way of conducting the research on their ‘home ground’. Each test was approximately 45 minutes in duration which included the time it took the participants to read out
53 | P a g e
loud the prescribed passages, as well as answers the subsequent questions verbally. Participants were frequently asked at the end of reading a passage if they would like to continue with the remainder of the test or if they would like to finish on their own terms. Identical testing conditions, environmental setting, procedure and time allocation was set up for the subsequent post-test at the conclusion of the 5-week study.
3.6.3 Instrument and Profile of Participants
The reading performance test administered to the participants individually, by the researcher, measured three traits of reading achievement; comprehension, accuracy and rate (words read per minute). The administered tests to both groups of participants were identical for each group and administered in the same environment and under consistent conditions. Both the pre and post-test were parallel in structure, consisting of six short reading passages. Subsequent to the participants reading a passage, four to five comprehension questions relating to the passage were asked. Reading passages and comprehension questions for the pre and post-test were of a similar skill but differed to eliminate familiarisation. The data obtained from the tests was used to answer the question;
‘Do iPads when used as an e-reader and application, in a middle school reading programme, influence student’s reading achievement?’
The two tests were administered to a total of 45 participants. The treatment group consisted of 19 participants (due to their access to an iPad while at school), while 26 were placed in the control group. The 45 participants from both groups combined made up for approximately 28% of the total student population within the middle school year group tested.
The pre-test was administered at the start of the investigation to all 45 participants, followed by the same 45 participants completing the post-test 5- weeks later
54 | P a g e
3.6.4 Conceptualisation of the Test
It is important that researchers who use published tests, are aware of the tests purpose, objectives and content aligning with that of their own during the evaluation, in essence, ”the test demonstrates fitness for purpose”. (Cohen et al, 2011, p.479).
The Neale Analysis of Reading Ability, 3rd Edition (Australian Standardisation) published by the Australian Council for Educational Research was used as the primary method of quantitative data collection within the quasi experimental pre-test-post-test non-equivalent group design.
The Neale Analysis of Reading Ability is an individually administered test of oral reading ability. The standardized test was chosen as it allowed the researcher to perform the small scale assessment in which the researcher assessed individual participants reading ability level in an informal one-on-on situation within the participants’ familiar and comfortable surroundings of the classroom. The tests were also utilized due to its inapplicability in New Zealand classrooms and therefore minimal chances of the participants’ prior exposure to the test before the commencing of the study. The testing procedure involved establishing the participants reading level followed by the participant progressively reading passages aloud and orally answering comprehension questions until a specified number of errors have been made. Each passage was set at a level which increased difficulty in vocabulary and grammar as the participant progressed. The researcher recorded the time taken in seconds and the errors made during reading on the student's individual record.
Upon completion of the test administration an error count of the number and types of errors was made (mispronunciations, substitutions, refusals, additions, omissions, and reversals) and recorded. The measures provide three raw scores: Accuracy, Comprehension, and Rate. The standardized forms of the reading
55 | P a g e
passages were presented in two parallel sets, allowing the researcher to assign one set as a pre-test and the other a post test.
Tests are frequently used in inter method mixing (Johnson & Turner, 2003). The inter method mixing of the standardised tests and a questionnaire was used within the study to provide information about what might affect reading achievement, Subsequent to the tests, participants completed a quantitative questionnaire in order to gauge their engagement and learning. The combination of the questionnaire and tests was fundamental to the research question of determining the relationship of engagement and beliefs to reading performance.
3.6.5 Limitation of Tests
“Users of test scores often assume achievement scores are direct and ambiguous measures of student achievement...” (Koretz, 2000, p.4) and previously, “the standardized achievement test score has been the operational definition for educational achievement and as such, raising test scores has been equated with educational improvement” (Haladyna, Nolen & Haas, 1991, p.2). However, scores in most achievement tests are limited to the measure of latent constructs of interest with certain aspects of student proficiency, (Koretz, 2000). Measures of such constructs tests can be seen as incomplete as they supersede to a fallible nature based on measurement error, their vulnerability to corruption or inflation (Koretz, Bertenthal & Green, 1999) and possible test score ‘pollution’. (Haladyna et al., 1991).
Yet, for the purpose of this research, it is noted that the test was used to the extent of allowing the researcher to justify their hypothesis in generalizing from the test scores to the latent construct of reading achievement.
56 | P a g e