MOVIMIENTO DE TIERRA

3.7.1 Collection of data concerning academic achievement

There is an examination at the end of each term for the high schools that I selected for my study, namely, the final-term examination. It usually incorporates various types of test question, including multiple-choice questions and short essay questions. The examination is usually criterion-referenced, and aims to test whether students have mastered the learning content taught during that term of study. One example from the English examination paper is as follows,

What else did you do _______ sending e-mails? ---I watched a TV series on CCTV-1.

A. besides B. except C. except for D. except that

This multiple-choice question examined whether students had mastered the grammar introduced in the textbook. Students’ abilities in analysing and solving problems are tested in the examination. One example was copied from the English examination paper, as follows.

From the report we can infer that Zhang Yan probably _____. A. is not sure of his success in the coming exam

B. does a very good job in his studies. C. will not be chosen by Peking University.

D. does not show much interest in Peking University.

This type of question was called reading comprehension. After reading a short article of about 300 words, students were asked to answer questions based on the article. The word ‘infer’ in the question was an indication that students’ ability to analyse, reason and draw implications was being tested. The final-term examination results were a good reflection of the learning outcomes that had been produced through participation in class during a certain period of time. All participants taking the examinations used the same examination papers, which allowed comparison between schools and classes. For each participant, I collected his/her scores in mathematics, physics and English produced by the final-term examination, which was used to represent his/her academic achievement in the corresponding subject. It is noticeable that some scholars have used not only the academic results that students achieved after classroom participation, but also those achieved before, which has allowed them to identify learning gains, for instance, the work of Brown and Palincsar (1989), and that of Fung and Howe

(2014). The literature reviewed in Section 2.5.1.3 suggests that previous academic achievement may have an effect on classroom participation. However, I only assessed students’ achievement on the basis of the final-term examination following their classroom participation, but did not ask for their scores before. This was because the students who constituted my research sample had just progressed through the division between scientific and liberal approaches. They were studying new courses from the second year, which was the time I conducted my fieldwork. The students who had chosen to focus on science would no longer be studying courses relating to liberal arts, and the scores achieved in the first year had little relevance to their second- and third- year learning. There were certain instances when students whose scores had been low in the first year turned out to be good in the second year, as they were then clear about their learning path following the scientific-liberal division. Thus considering their previous academic scores, namely, those achieved in the first year, were less necessary in my study. Moreover, it was almost impossible to get their previous academic scores, as the participants were in different classes in the first school year and formed new classes from the second school year. The participants’ previous scores were needed to pick out one by one, if their prior achievement needed to be assessed. Given a large number of students involved in my study, it would take ages to obtain their previous scores and the schools would not do it.

3.7.2 Collection of data concerning cognitive abilities

There are two recognized instruments that have been designed in order to assess the analytical, creative and practical abilities delineated from the Sternberg’s Triachic Theory. One is called the Sternberg Triachic Ability Test (STAT) designed by Sternberg (1993), and the other one is the Self-Rated Ability Scale (SRA scale) proposed by Zhang (1996). I employed the STAT (Sternberg, 1993) to assess the analytical, creative and practical abilities proposed in Sternberg’s (1985) Triarchic Theory. There were three main reasons for using the STAT. Firstly, it has been used in various cultures, such as Finland, the United States and Hong Kong, and has proved to be reliable and valid (e.g. Sternberg, 1999; Sternberg & Grigorenko, 2007). Secondly, it is the one of Sternberg’s instrument that has been used when scholars have previously investigated the relationship between thinking style and abilities (see Zhang, 2004, 2010). I considered I would be more convincing if I used the same tool as previous studies had and based my assumptions on their results. Thirdly, in contrast to the STAT, the instrument designed by Zhang (1996) relies on students’ self-report. This type of measurement is easily affected by students’ personal factors and may be subjective in accounting for students’ ability. Students’ self- rating has been found to be affected by their self-esteem (Felson, 1981; Wells & Sweeney, 1986). It is possible that two students with the same level of cognitive ability may self-rate themselves as different,

as one student may have a higher level of self-esteem than the other. In order to maintain the objective and fair evaluation of cognitive ability, Zhang’s instrument was less suitable and I thus employed the STAT instrument designed by Sternberg.

The STAT is a performance test, in which Level H is designed for students aged 16 and above. There are both Chinese and English versions. The STAT consists of both a multiple-choice test and an essay test to measure abilities, yet most studies employ the multiple-choice test only since the essay test cannot be graded objectively (see e.g. Sternberg, Castejón, Prieto, Hautamäki, & Grigorenko, 2001; Zhang, 2004). I also used the multiple-choice test only, which consists of 36 items, with 12 measuring each ability. In order to offer greater opportunities for students to show their abilities (Sternberg, 1999), Sternberg measures each ability in relation to three domains – verbal, quantitative and figural. This yields nine subscales (three abilities × three domains): analytical-verbal, analytical-quantitative, analytical-figural, creative-verbal, creative-quantitative, creative-figural, practical-verbal, practical- quantitative, and practical-figural.

Testing of each aspect of intelligence is done through three modes of presentation of problems: verbal, quantitative, and figural. Thus, analytical intelligence is measured through tasks relating to the application of meta-components, performance components and knowledge-acquisition components to conventional verbal, quantitative and figural problems requiring analysis and evaluation. Creative intelligence is evaluated by means of problems whose solutions require ‘insight’ processes for verbal, quantitative and figural contents. Practical intelligence is measured by applying these components to verbal, quantitative and figural problems that relate directly to everyday life. Examples of the STAT are illustrated in Appendix D.

This assessment tool is still at an experimental stage, but it has been used in a variety of cultural contexts, including the United States (Sternberg et al., 2001), Finland (Hautamäki, Arinen, Hautamäki, & Scheinin, 1998), Hong Kong (Zhang, 2004, 2010) and Spain (García, 1997). The STAT has been proved to have satisfactory construct validity. For instance, Sternberg et al. (2001) conducted a hierarchical confirmatory factor analysis to examine the structure of the Triarchic Theory of Intelligence, and the model’s fit turned out to be generally good. Moreover, studies have reported that the STAT generally has satisfactory internal consistency. As reviewed by Zhang (2010), the overall Cronbach’s alphas in all social contexts are above .60. It is noticeable that, in a recent study, Zhang (2010) has used the Chinese version of STAT to collect data from secondary school students in Hong Kong. The Cronbach’s alphas were .67, .70 and .36 for the analytical, creative and practical abilities respectively. The alpha for the practical ability was below .60 and thus did not qualify for use in subsequent analyses. This may due to the fact that Zhang focused on secondary school students. Secondary schools in Hong Kong generally pay less attention to connections between knowledge and

real life (Cheung & Yip, 2003). The students there have limited access to practice and may therefore have a less mature understanding of issues in practice, and thus may become confused when they are tested with practical questions.

In my study, the measurement of cognitive ability was conducted once at the end of the Autumn term. The STAT is quite long and complicated, requiring participants to answer it carefully. I obtained assistance from the principals and teachers in the schools. They organized students to fill in the STAT during a particular period of time. This ensured that the students were focused on the test, and not distracted by other things. Also, with the teachers standing by, there was no chance of the students copying from each other. It generally took the students one hour to finish answering questions in the STAT and then I collected their test papers for grading. The STAT mark that each student achieved was capable of reflecting his/her real level of cognitive ability. I intended to conduct the measurement twice when I designed the research: one at the beginning of the term and the other one at the end, which was expected to allow me to see the changes in students’ abilities following their classroom participation. However, I found that it was not realistic in practice, as organizing students to take the test was time-consuming and required assistance from the school. Students had a heavy study load, and their time was precious, which made it hard for me to obtain agreement for the measurement to be conducted on two occasions. Further, almost all studies conducted by other scholars measure cognitive ability once (e.g. Sternberg, 1997; Zhang, 2010). Thus in my study, students’ cognitive ability was also measured once at the end of the term.