Usuarios, productores, formadores Usuarios

Self-assessment of second or foreign language abilities is defined as “language learners’ evaluation of his or her own language skills, usually in connection with a language course or part of other forms of language assessment” by Luoma (2013). Self-assessments are usually made with explicit reference to clear learning goals in a target domain. Therefore, self-assessments

could be another potential external criterion used in the study of the extrapolation inference, In addition, self-assessments can be a reliable learner-directed measure of foreign language or second language proficiency that brings test-takers’ voices to the validation process (LeBlanc & Painchaud, 1985). As for the efficacy of self-assessment by naive or untrained learners, research shows that self-assessment is feasible even for young learners. For example, the correlation between self-assessment of reading skills made by 5271 third graders in Sweden was found to be .58 with a standardized test of reading skills and .59 with teacher judgment (Johansson, 2013). The self-assessment reported in that study contained four statements on a 4-point Likert scale. Johansson (2013) also found that gender and socioeconomic status did not affect third-graders’ self-assessment, which, together with the correlation coefficients, leads to his claim that for these young learners, self-assessment of reading literacy skills was a reliable assessment tool.

In terms of data collection and accessibility, self-assessment is easier to use and could reach out to a larger number of targeted participants with a set of uniform or standardized tasks/items and/or rubrics, compared to teacher evaluation and peer assessment. Another advantage of using self-assessment is that it is convenient and less time-consuming to ask students to directly reflect on or evaluate their abilities in a given context (Coronado-Aliegro, 2006). Self-assessments are usually administered in a low or non-threatening environment with minimal consequences. As mentioned in Powers et al. (2009), self-assessment tends to be more valid in the cases where participants have “no incentive to intentionally distort their reports” (p. 15).

Self-assessment has been recognized as an important instructional and learning

instrument in the field of education, where the major interests are the facilitative effects of self- assessment on learning outcomes and the relationship between self-assessment and other

individual difference constructs, such as self-regulated learning strategies (Kostons, van Gog, & Pass, 2012; Lew, Alwis, & Schmidt, 2010; Ross, 2006). The importance of self-assessment for learning has been widely acknowledged for its positive effect on learner awareness and learning outcomes (Luoma, 2013). Self-assessment can be made in multiple formats, such as open-ended questions, checklists, and can-do statements with a Likert-scale. Self-assessment also has its value as an alternative to some exiting tests or as a tool to validate a test (Alderson, 2005; Engelhardt & Pfingsthorn, 2013; Hellekjær, 2009; Lee & Greene, 2007).

The utility of self-assessment of English skills has been explored mainly via correlational analyses with other measures, for example, scores on standardized English proficiency tests and teacher ratings. Overall, the findings about the correlation between self-assessment tools and other measures are promising, although the magnitude of correlation coefficients varies from study to study depending on the item format and specificity of item content (Brantmeier, 2006; LeBlanc & Painchaud, 1985; Luoma, 2013; Ross, 1998). LeBlanc and Painchaud (1985) used a planned self-assessment questionnaire as a placement tool, which contained 60 “can-do” statements with reference to specific situations. They found that the self-assessment tool produced high quality results and placed students in a similar way as the standardized tests did. Malabonga, Kenyon, and Carpenter (2005) investigated the relationship between university students’ performances on a self-assessment and a computerized oral proficiency test of foreign language. It was found that 98% of the students in that study could successfully use the self- assessment to select the test tasks that were appropriate to their foreign language proficiency levels. In addition, the correlation between the self-assessment and teacher ratings of oral proficiency ranged from .74 to .81. Strong-Krause (2000) proposed a use of self-assessment for

ESL placement purposes. Other studies also suggested or examined the use self-assessment for similar purposes, especially in writing assessment (Gere et al., 2010).

Considering self-assessment as an instrument to elicit individuals’ responses with reference to relevant real-life tasks, a series of self-assessment-based validation efforts has been made in recent years, especially for the Test of English for International Communication

(TOEIC) (Powers, Roever, Huff, & Trapani, 2003; Powers, Kim, & Weng, 2008; Enright, Bridgeman, Eigorn, Lee, & Powers, 2008; Powers, Kim, Yu, Weng, & VanWinkle, 2009; Ito, Kawaguchi, & Ohta, 2005). As Powers et al. (2008) pointed out, “one kind of evidence that has proven useful in elucidating the meaning, or validity, of language test scores has come from examinees themselves, in the form of self-assessments of their own language skills” (p. 8). In Powers et al. (2008), 49 can-do statements about English reading and listening in everyday life activities were administered to 7,292 test-takers in Japan and 3,636 test-takers in Korea

immediately after the TOEIC test. It was found that the self-assessment had a relatively strong correlation with the TOEIC scores, ranging from .47-.53. In another research report, Powers et al. (2009) attempted to link test-takers’ performance on the speaking and writing sections of the TOEIC test with test-takers’ self-evaluation of real-life activities. Forty can-do statement about speaking tasks and 29 can-do statements about writing tasks were administered to 2,947 test- takers in Japan and 867 test-takers in Korea. The correlation coefficients between the self-

assessment and the TOEIC scores ranged from .52 to .54. Powers et al. (2008) concluded that the use of self-assessment provided evidence for the linkage from test performance to test-takers’ performance in a variety of English-related activities in real life and that the can-do statements in the self-assessment can be “reasonably trustworthy validity criteria” (p. 12). Ito et al. (2005) studied 8,386 Japanese company employees’ self-assessment of functional job performance and

compared the self-assessment with their TOEIC scores. Their self-assessment consisted of 65 five-point Likert scale can-do statements related with job activities in seven situational domains. It was found that the self-assessment was highly correlated with the TOEIC scores, with the correlation coefficient ranging from .62 to .71.

Self-assessment was also used to represent test-takers’ real life experiences in an academic context in a recent criterion-referenced validation study of the General English Proficiency Test (GEPT) in Taiwan (Weir, Chan, & Nakatsuhara, 2013). However, the relationship between the GEPT test and test-takers’ self-assessment was not reported due to some technical issues in their study. Instead, Weir et al. (2013) focused on the relationship between the GEPT test and the IELTS test as well as that between the GEPT test and some real life academic performance on course assignments and examinations.

In a validation study of the TOEFL iBT, a self-assessment, along with academic placement and instructor’s ratings, was used as a piece of evidence for the extrapolation inference (Enright, Bridgeman, Eignor, Lee, & Powers, 2008). Using confirmatory factor analysis, Enright et al. (2008) identified four factors corresponding to the four sub-skills (reading, listening, speaking, and writing) in the self-assessment. The four factors in the self- assessment were found to have a moderate and positive correlation with test-taker’s performance on both TOEFL PBT and the prototype measures of the TOEFL iBT, with the correlation

coefficients ranging from .30 to .62. Enright et al. (2008) regarded the magnitude of correlation to be “high” and “similar in magnitude to other test-criterion relationships” (p. 178).

However, like standardized English proficiency tests, self-assessment is not without problems when used as an external criterion for validation purposes. There are several factors that can influence individual’s self-assessment, including the characteristics of self-assessment

items and personal traits. Zell and Krizan (2014) in their meta-synthesis of 22 meta-analytic studies on self-assessment in various fields identified five moderators between self-assessment and other measures: performance domain, academic discipline, task objectivity, task familiarity, and task complexity. In terms of task characteristics of self-assessment, Butler and Lee (2006) compared off-task and on-task self-assessment tools in measuring the oral English proficiency of the fourth grade and sixth grade Korean learners of English. The two types of self-assessment were similar content, but differed with regard to the time of administration and reference to learning tasks. It was found that on-task self-assessment had a higher correlation with general proficiency test scores and teachers’ assessment. In addition, on-task self-assessment appeared to be less prone to the influence of attitudinal and personality factors. Their findings about the differences between the two types of self-assessment echo with the research on the effect of task specificity in self-assessment (Oscarson, 1997; Pajares, 1996; Zell & Krizan, 2014; Strong- Krause, 2000). In a meta-analysis of self-assessment studies in the field of language testing, Ross (1998) identified that domains could be a factor contributing to the variability of the correlation between self-assessment and other measures. For example, self-assessment of receptive skills (reading and listening) tended to have a higher correlation with other outcome measures in the meta-analyzed studies than that of productive skills (speaking and writing).

Of course, self-assessment instruments are not without problems. The accuracy in self- assessment in general has been questioned as self-perceptions are sometimes found to be “off the mark”, conflicting with external judgment in certain domains (Zell & Krizan, 2014, p.112). This could be accounted for by research participants’ experience in evaluating themselves and the influence of participants’ social desirability, especially in sociological and psychological research.

Overall, comparing the scores from English placement tests with those from standardized English proficiency tests and self-assessments should shed light on the plausibility of the

extrapolation inference; even they might not be optimal external criteria of the English proficiency in the target domain.