Características morfosintácticas

Achievement tests are generally used to measure the learning achievement levels of students. They can be constructed by the teacher or the researcher or through collaboration. The main objective of these achievement tests is to serve the specific research needs of researchers. However, these tests are rarely of high quality as compared to large international tests such as PISA and TIMSS (Hambleton, 2014). In some situations, achievement tests measure more than cognitive achievement. Achievement tests can also take the form of performance tests, which require students to perform (to construct a diagram, map, writing responses) some skills (Hambleton, 2014). Furthermore, these tests can either be norm-referenced or criterion-referenced in nature.

A norm-referenced test assesses a construct such as learning according to set scoring norms and usually compares students, whereas a criterion-referenced test assesses each student in terms of fulfilment of pre-defined criteria (Cohen, Manion, & Morrison, 2007). Hambleton (2014) comments that criterion-referenced tests are suitable for a study where the researcher is interested in assessing the levels of accomplishment of participants, as is the case of the current study, which was to determine advanced, proficient, intermediate and novice levels of achievement. These tests provide the researcher with rich evidence “about exactly what a student has learned” (Cohen et al., 2007, p. 416) and can indicate whether or not a given student achieved the set criteria. Such tests can be used to monitor learning progress and assess gains over a period of time, and may thus serve as formative assessment tools, for instance, serving the learning purpose.

Given the focus on assessment of students’ conceptual learning, the criterion-referenced Achievement Test in Science for grade 9 (ATS9; appendix H) used in the current study was designed and developed to assess students’ science proficiency levels. Because part of Research Question One required assessment of students’ accomplishment in terms of achievement scores, the ATS9 also served the purpose of a norm-referenced test to compare students’ achievement across different classrooms. However, the overall objective behind the construction of ATS9 was to measure learning proficiency levels and analyse them with reference to other research tools.

The ATS9 was constructed based on the learning objectives and content (concepts) outlines of the five chapters from the science textbook (see notes for table 3.1), which were covered during the 10 weeks of the teaching experiment. The revised version of Bloom’s taxonomy (Krathwohl, 2002) was followed while designing the ATS9 items. The first three out of the four dimensions of knowledge, viz. factual, conceptual, procedural and metacognitive knowledge (figure 3.7) as suggested by Krathwohl (2002), were used to design the test items. The fourth aspect of meta-cognitive knowledge was dropped because the focus of research was on cognitive dimensions/skills only. An explicit focus on the development and measurement of the lower-order cognitive skills (LOCS) and the higher-order cognitive skills (HOCS) (Bramwell-Lalor & Rainford, 2013) was made explicit for the purpose of teaching, learning and assessment (see figure 3.7). The data obtained from the posttest scores of ATS9, is used to answer the

first part of research question one (section 2.9) and the results obtained for the conceptual learning were used to answer the second part of research question one of this study.

The ATS9 (see appendix H) contains a total of 35 items, of which 12 are multiple choice items and the remaining 23 items require students to write open ended responses. Out of these 23 open ended questions 13 items are identified as LOCS and 10 items as HOCS. The LOCS items were classified in terms of three categories: Proficient, Intermediate and Novice, while the HOCS items were categorized in the four categories: Advanced, Proficient, Intermediate and Novice. Figure 3.6 shows the focused area of knowledge and cognitive dimensions of the ATS9 in relation to the development and assessment of LOCS and HOCS of students. Details of how each assessment item is delegated to the knowledge dimensions in ATS9 are presented in tables 3.1 and 3.2.

The ATS9 was pilot tested with grade 9 as well as grade 10 students to establish validity and scoring reliability based on the set criteria for scoring. Face validity of the ATS9 was ensured by conducting group discussion involving the participant students and teachers during pilot testing. The feedback provided by students and teachers helped the researcher in refining some of the items. Content validity was maximized by discussing ATS9 items with two science teachers and two teacher-educators who generally design

Focused knowledge dimensions and cognitive process dimensions

area

T hinking skills L ower

order

H igher order

Figure 3.7 Focus of the Achievement test based on the revised Bloom's Taxonomy (adapted from Krathwohl, 2002)

achievement tests for different examination boards. Most of the ATS9 items were compared to some of the achievement tests previously used by one of the examination board. To ensure reliability of the results, the ATS9 was applied to grade 10 students from two different schools along with a group of grade 9 students. Different responses obtained for the ATS9 items were compiled to ensure scoring reliability. The ATS9 assessment rubric (appendix I) and scoring criteria was established by the participant science teachers in collaboration with the researcher. Achievement scores were allocated to different ATS9 responses according to the defined categories of levels of conceptual learning.

Table 3-1 ATS9 Blue Print: Chapter wise distribution of items according to cognitive process dimensions (adapted from Krathwohl (2002))

Chapter* Remember Understand Apply Analyse Evaluate Create Total**

3 and 4 (chemistry) 4 # _{21, 22, 25 2, 23} 1, 3, 11, 12, 13, 14, 24 13(18) 7 (life sciences) 16 27,28 5, 6, 15, 26 7(10) 9 and 11 (physics) 8 17, 18, 19, 20, 31, 32, 35 7, 9, 10, 33, 34 29, 30 15(22) Total 3(3) 12(20) 2(3) 16(20) 2(4) 35(50)

Notes: * Chapter3: Structure of the Atom, Chapter 4: Atoms and Molecules, Chapter 5: Diversity in organism, Chapter 9: Work, power and energy, and Chapter 11: Sound.

** Total possible achievement scores are shown in brackets with number of items for different chapters and cognitive process dimensions in the last column and last row: number of items (total score)

# indicates respective question number in the ATS9

Table 3.1 presents the distribution of items of the ATS9 according to the cognitive process dimensions and the chapters covered from the grade 9 science curriculum during the intervention period. The last column of the table shows the total number of items from different chapters and their corresponding achievement scores in parentheses. For example, from the two chemistry chapters a total of 13 items were designed which make up of a total of 18 marks (shown in brackets), out of the total 50 marks for the

achievement test. Similarly, the last row of Table 3.1 presents the total number of items and maximum marks allotted each cognitive process domain.

Likewise, Table 3.2 presents the distribution of items and their respective scores in parentheses allocated according to the knowledge dimensions of factual, conceptual and procedural knowledge. The last column of the table shows the total items that assess a particular kind of knowledge and the total possible score a participant could achieve. The last row of the table shows the total number of items and their corresponding scores according to the six cognitive processes of Krathwohl’s (2002) revised Blooms taxonomy of instructional objectives.

Table 3-2 Taxonomy table for ATS9: Knowledge and cognitive process dimensions (adapted from Krathwohl (2002))

The Knowledge Dimension

Remember Understand Apply Analyse Evaluate Create Total

Factual Knowledge 8 # _{5, 6, 9 4(5)*} Conceptual Knowledge 4, 16 17, 18, 19, 20, 21, 25, 27, 28, 31, 32, 35 1, 3, 10, 11, 15, 24, 26 20(28) Procedural Knowledge 22 2, 23 7, 12, 13, 14, 33, 34 29, 30 11(17) Total 3(3) 12(20) 2(3) 16(20) 2(4) 35(50)

Note: # indicates respective question number in the ATS9 * Total number of items and their respective possible total scores

The ATS9 (also served as a performance test) provided samples of students’ work which were analysed for conceptual learning and compared with students’ group concept maps (the following sub-section). Students’ individual proficiency levels were evaluated against the group concept maps to assess their individual as well as group conceptual learning/understanding. Anderson and Krathwohl’s (2001) revised Bloom’s

taxonomy: A taxonomy for learning, teaching, and assessing, was used to design the ATS9 items (Tables 3.1 and 3.2) as well as to guide the assessment process.