Those scholars who cast doubt on whether or not critical thinking can be defined also doubt whether or not it can be assessed, claiming that such a vague concept cannot be measured (Pithers and Soden 2000). In this research, however, the opposite position was assumed, since, as shown in the discussion in section 2.2.1, it is possible to define the concept. Effective assessment is important for research into the teaching of thinking since it will contribute to the validity of measurable results. It also allows for a comparison of results within and between groups.
Some assessments measure critical thinking by tests involving multiple-choice questions. The Watson-Glaser Critical Thinking Appraisal (WGCTA) is distinguished by its long history, having been first designed in 1937, with US and UK versions being further developed in 1980 and 1991 respectively (Hassan and Madhum 2007). Although many studies of college students have benefited from its contribution in providing valid and reliable results (Hergovich and Arendasy 2005; Brown et al. 2001; Girot 2000), one of its limitations is that it is concerned solely with the ability to think critically, and fails to investigate the disposition to do so (Ku 2009).
Facione et al. (1994) developed a set of tests to evaluate both critical thinking skills and relevant dispositions, including the California Critical Thinking Skills Test (CCTST) and the California Critical Thinking Disposition Inventory (CCTDI). The former is a multiple-choice test, while the latter uses six-point Likert scales. These have been widely used in college (Blondy 2011; Wangensteen et al. 2010; Ozturk et al. 2008; Yang et al. 2008; Raymond et al. 2005) and high school (Miri et al. 2007) studies, and
30
have been proven to be valid and reliable (Phillips et al. 2004; Facione et al. 1994). Both the CCTST and CCTDI have been translated into Chinese and were revised by Luo and Yang in 2002 and 2001 respectively. The Chinese versions have also been used in studies of college students (Liu and Zhao 2010; Luo and Yang 2001) and high school students (Zhou et al. 2012; Qing et al. 2010; Zhou et al. 2007), with the results also supporting the validity and reliability of these tests. However, these tests also have their limitations (Facione et al. 1995). On the one hand, the CCTST may not motivate test-takers to engage in deep thinking. They can complete the test by guessing, and still arrive at correct answers. Takers of the CCTDI test may understand the purpose of the test and select desired responses in order to get high marks. In such cases, the CCTST can fail to reveal test-takers’ actual ability in critical thinking, and the CCTDI can fail to reveal actual dispositions.
Facione et al.’s measurements of critical thinking are valuable for gaining insight into
the disposition to think critically, since, as many studies have suggested, critical thinking disposition is a significant predictor of cognitive development (Rapps et al. 2001), school performance (Ip et al. 2000) and the development of critical thinking (Facione 2000; Facione et al. 1995). Rapps et al.’s (2001) study investigated four factors influencing cognitive developments and the results revealed that only critical thinking disposition was able to predict all the levels of cognitive development targeted. Ip et al.’s (2000) study suggested a significantly positive correlation between the
strength of critical thinking dispositions and grade point averages (GPA). Facione et al.’s study (1995) revealed a greater development in critical thinking on the part of
31
university than those with a weaker disposition on entering. Therefore, assessments which target both skills and dispositions are useful.
Other researchers began to assess critical thinking through the assessment of written texts produced by the participants. According to Lantolf (2006), writing is a way of vocalising or revealing cognitive activities and processes and thus makes them recordable; it is therefore an indication of whether or not learners have used critical thinking in real practice and of how they have used it. The Ennis-Weir Critical Thinking Essay Test (Ennis and Weir 1985) was designed for use with high school and college students and has been used in many studies (Williams and Worth 2009; Clifford et al. 2004; Yeh 2001; Dunham 1997). It targets students’ ability to judge and
formulate arguments, and the results reflect to some extent their ability in and disposition towards using critical thinking (Ku 2009). One limitation of the Ennis-Weir test is that it may not be able to reveal the actual critical thinking ability of EFL and ESL learners, especially those from Asian countries, since it was designed for native English speakers (Dunham 1997). The topics included may not be familiar to many EFL and ESL learners. Stapleton (2001) claimed that lack of familiarity with the topic of writing could restrict students in demonstrating their critical thinking ability.
Stapleton (2001) then proposed a model to identify key elements of critical thinking in argumentative writing in which the topic can be selected and designed by the researchers. These elements include an argument, defined as a claim with a supporting reason, evidence, opposing viewpoints, refutations and conclusions. Using this model, researchers can take the educational background, age and language level of writers into account, and then select more suitable topics for them to display their critical thinking.
32
This model has been adopted by Alagozlu (2007) and is deemed to be a useful tool. However, more studies are needed to examine its reliability and validity.
In this study these views were combined, and the students’ thinking was investigated using both the California tests and assessments of the students’ written work. The aim
was to assess both critical thinking skills and dispositions, and at the same time to investigate whether and how the students used critical thinking in their learning practice. This combination was expected to enhance the validity and reliability of the results of the present study by allowing the results of the two methods to corroborate each other (for further discussion see chapter 3, section 3.4.2).