Principios de resolución de disputas (estándares)

A perusal of the literature on teachers’ cognition in relation to assessment reveals a great diversity of research studies carried out from 2003 to 2017. Although none of these studies use the term cognition to refer to their participants’ mental representations, most studies resort to other terms such as beliefs (Barnes, Fives, & Dacey, 2015; Büyükkarcı, 2014; Chang, 2006; Elshawa, Abdullah, & Rashid, 2017; Gutbrie, 2005; Karim, 2015; Mansory, 2016; Restrepo & Aristizábal, 2003; Sikka, Nath, & Cohen, 2007; Thomas, 2012), conception (Deneen & Brown, 2016; Opre, 2015) and/or perception (Ağçam & Kaya, 2017; Assunção, 2017; Figueiredo, Alves, & Silva, 2016; Mussawy, 2009; Sahinkarakas, 2012). In addition, studies on teachers’ cognition in relation to assessment have been conducted in a number of different contexts, including Turkey (Ağçam & Kaya, 2017; Büyükkarcı, 2014; Sahinkarakas, 2012), Malaysia, Taiwan and Fiji (Chang, 2006; Elshawa et al., 2017; Opre, 2015), America (Restrepo & Aristizábal, 2003; Sikka et al., 2007), Saudi Arabia, Pakistan and Afghanistan (Mansory, 2016; Mussawy, 2009; Thomas, 2012), among others.

It is also apparent that a large body of work in L2 teacher cognition concerning assessment tends to focus on various issues related to classroom assessment in general and are not topic- specific. For example, many studies encourage investigating teachers’ beliefs about formative assessment (Büyükkarcı, 2014; Karim, 2015; Mehrgan, Candidate, & Language, 2017;

Thomas, 2012), while the majority of work still advocates comparing pre-service teachers’ beliefs to in-service ones, considering the role of experience, training and assessment literacy as the main factors that shape their participants’ beliefs (Assunção, 2017; Büyükkarcı, 2014; Deneen & Brown, 2016; Mehrgan et al., 2017; Mussawy, 2009; Sahinkarakas, 2012; Sheehan & Munro, 2017).

In this section, I have identified previous studies that are most relevant to teachers’ beliefs about assessment. In the following sections, I review the relevant aspects of these studies and discuss them in more detail.

2.4.5.1 Beliefs about the purpose of assessment.

Brown (2002) devised a Teacher’s Conceptions of Assessment (TCoA) inventory providing four main purposes upon which the conception of assessment revolves around. These conceptions are as follows:

• It improves teaching and learning.

• It makes students accountable for their learning.

• It holds schools and teachers accountable for their students’ learning. • It is irrelevant, invalid and should be rejected.

Brown’s first conception implies that assessment can serve as a tool to diagnose students’ learning strengths and weaknesses. Therefore, teachers should employ forms of assessment that allow them to better identify what students learn and what they fail to comprehend. This would enable teachers to target the less-comprehended topics again later in the courses and provide the learners information about what they particularly need to work on regarding learning. The second conception, student accountability, means that the assessment prompts students to be responsible for their own learning and make more efforts by providing them with a measure of what they have achieved so far. The third conception (teacher and school accountability) refers to the use of assessment as evidence to prove that teachers and schools, or in my case the higher-educational sector, complies with the established standards and keeps both sides motivated. Last, ‘assessment is irrelevant’ means that assessment is seen as purposeless, perhaps because the teachers know their students and the specific curriculum,

and assessment would place pressure, affecting the teachers and the students negatively (Brown, 2002, p. 41).

Using TCoA, Barnes, Fives and Dacey (2015) placed these conceptions on a continuum where, at one end, assessment plays an extreme pedagogical role focusing on students learning and, at ‘the opposite extreme [,] assessment reflects the sole purpose of high-stakes accountability’ (p. 286). Some conceptions as to the purpose of assessment may be located in the centre of the continuum (mixed) because they combine both pedagogical and accounting purposes; how close they are to one end or the other is determined by the way in which an assessment is embedded in a specific context. Outside the continuum, the researchers have taken the irrelevance view, which is associated with assessment being considered as bad and to be ignored. ‘Thus, if teachers believe that assessment is irrelevant then it cannot be used for any of the purposes along the continuum’ (p. 291). To clarify this continuum and the components at each end, I have devised the following figure.

Figure 6. Continuum of beliefs and conceptions about the purpose of assessment. (Barnes, Fives, & Dacey, 2015, p. 287)

While Brown’s TCoA inventory provides a useful framework for teachers’ conceptions of assessment, it has its limitations. Harris and Brown (2009) argue that the TCoA inventory constrains the purposes of assessment to these four conceptions, while leaving out other significant purposes that may be captured by other studies. In their study, seven major purposes of assessment are identified: (1) compliance, (2) external reporting, (3) reporting to parents, (4) extrinsically motivating students, (5) facilitating group instruction, (6) teacher use for individualising learning and (7) joint teacher and student use for individualising learning (p. 369).

These seven purposes listed above, however, may readily be considered as separate

subcategories that fall within the four broad categories of the TCoA (see Figure 6), and they elaborate on the account which TCoA provides. Thus, I believe that Harris and Brown’s argument is weakened and Brown’s TCoA remains valid at a more general level, as a guide for research studies in the field of teachers’ beliefs regarding assessment.

Empirical studies on teachers’ beliefs and perceptions about the purposes of assessment point to a degree of agreement among teachers’ conceptions across contexts in that they tend to frame assessment as a vital instrument that is employed to achieve various pedagogical and educational goals (Ağçam & Kaya, 2017; Barnes et al., 2015; Chew & Lee, 2008; Elshawa et al., 2017; Mussawy, 2009; Opre, 2015; Sikka et al., 2007).

For example, Elshawa et al. (2017) investigate the assessment beliefs of ESL instructors at a tertiary level in Malaysia, including purposes, methods, feedback and grades reporting. Adopting a cross-sectional research design, the researchers selected their participants purposively. 83 university teachers from six universities answered a four-point Likert-scale

questionnaire composed of 67 items that addressed various aspects of assessment, 10 of which concerned beliefs about assessment purposes. The results revealed that the participant- teachers believed that assessment fulfilled two main pedagogical purposes: informing

instruction and improving learning. The focus was, however, subtly different from my own. First, the word ‘assessment’ was not defined and probably was taken by the participants in a broader sense than I intend to use it (just limited to classroom-based assessment). Second, in the text, the researchers refer to measuring teachers’ beliefs and say, for example, that the teachers agreed that ‘assessment should be used for different purposes’ (p. 32), which appears to match my focus on belief in the sense of espoused theory, as described in 2.3.1.1.1. In reality, however, the items offered in the questionnaire for the participant teachers were not worded in that form, e.g. ‘Assessment motivates my students to learn’ (p. 44). Such items try to elicit teachers’ reports about what they think actually happens (i.e., practices), instead of their beliefs about what should be happening. Hence, Elshawa et al.’s (2017) article exhibits some conceptual confusion over which construct it is really studying; this is something that I intend to avoid, as it does not, in the end, precisely fall in my area of interest.

In another study, the analyses of prospective teachers’ perceptions of assessment and evaluation were similar to Brown’s TCoA inventory (Ağçam & Kaya, 2017). Using a questionnaire comprising both open-ended and closed items, the researchers compared two groups of prospective teachers studying classroom teaching at a state university in Turkey. This study included practising teachers who did not actually set assessment tasks themselves but rather were the subjects of assessment. The first group was composed of participants who had not taken the course titled ‘Assessment and Evaluation’, while the second group included those who had taken the same course as part of their second-year studies at university.

assessment and assessment practices in higher education. Third-year participants

acknowledged the necessity for assessment as a tool that provides learners with feedback about their strengths and weaknesses (an extreme pedagogical purpose). The first-year participants, on the other hand, believed that assessment is unnecessary and irrelevant. This attests to the impact of training, through a course which essentially changed teachers’ content knowledge of assessment.

Pereira and Flores (2017) report how teachers view assessment in higher-education levels after the implementation of the Bologna Process, an agreement issued in 1999 onwards to

ensure the comparability of standards and the quality of higher-education qualifications

across European countries. 57 teachers from five Portuguese public universities participated in the study. Data were collected via face-to-face interviews and online open-ended

questionnaires. The findings showed that teachers’ perceptions of the purpose of assessment remain primarily pedagogical, in that the role of assessment was perceived as to provide feedback that would allow students engage better with their learning processes.

Furthermore, two qualitative studies have identified teachers’ conceptions of assessment. Deneen and Brown (2016) interviewed 32 pre-service and practicing teachers about assessment, ranging from understanding and interpreting large-scale test results to the effective use of AfL techniques inside classrooms. Their results revealed polarized views stating that assessment carries both positive and negative consequences. Positive conceptions about assessment indicated that assessment helps students improve their academic capacities and boosts their self-esteem. The negative responses, however, were more related to non- academic aspects. For example, some of the participants mentioned that assessment results may make students feel embarrassed or bad. Moreover, the participants expressed some

concerns about the aspect of assessment fairness. Similar concerns were found in other studies where the participant teachers stressed that assessment should be fair to all students (Saad, Sardareh, & Ambarwati, 2013; Munoz, Palacio, & Escobar, 2012). In addition, Deneen and Brown’s study reported that ‘none of the participants considered assessment to be irrelevant’ (p. 8).

Dayal and Lingam (2015) explored in-service and pre-service teachers’ conceptions of assessment in primary and secondary schools in Fiji. Data collected through a reflective exercise with open-ended items were analysed qualitatively. The findings of this study were similar to the results reported in the previous research. Teachers from both groups had distinctive opinions about the primary purpose of assessment. In-service teachers’

perceptions about assessment were more inclined towards the extreme pedagogical end of the scale. These teachers believed that assessment is meant to point out the strengths and

weaknesses of teaching and learning strategies, leading to improved teaching and learning. Pre-service teachers, on the other hand, focused on the summative function of assessment, such as providing students with scores and measuring how much knowledge students possess subject-wise. In general, all the participants from both the studies were found to hold distinct and contrary beliefs about the purpose of assessment, all of which fall along the continuum of purposes captured in Figure 6.

Other studies have also reported conceptions of assessment at the extreme pedagogical ends of the aforementioned continuum, e.g. the conceptions of 25 lecturers from five public universities in Iraqi Kurdistan, who believed that assessment is used to improve learning and provide feedback on learners’ strengths and weaknesses (Karim, 2015). The belief that

by 69 English language teachers in different state primary schools in Adana in Turkey (Büyükkarcı, 2014). This conception was further supported by other participants in relevant studies who agreed that assessment improves teaching and learning by providing constructive feedback (Muñoz, Palacio, & Escobar, 2012; Saad et al., 2013; Sahinkarakas, 2012; Thomas, 2012).

Moving towards the other end of the continuum, cases where teachers’ conceptions of assessment reflected accountability purposes were also found in some studies. Davis and Neitzel (2011) interviewed 15 upper-intermediate and middle school teachers of two schools in the south-eastern part of the United States. They found that the majority of teachers’ conceptions about assessment was geared towards satisfying external audiences, namely the parents. Indeed, they believed that assessment required them to report to their students’ parents and impart information about their children’s progress on skills; for them, assessment would primarily serve a purpose of accountability.

Harris and Brown (2009) again reported teachers focusing on a similar function of

assessment in relation to reporting to external audiences. 161 five-ten-year-old teachers from

36 schools in the Auckland region participated in their study. 26 teachers were then selected for interviews based on their different perception profiles. After analysing the data, the researchers reported seven categories related to the purposes of assessment, as described earlier in this section: ‘Three categories (External reporting, Reporting to parents,

Extrinsically motivating students) were all related to accountability’ (p. 377). The majority of the teachers expressed agreement that assessment serves as evidence of achievement that can be presented to external organizations (e.g. Ministry of Education, school boards etc.). Reporting to parents was also considered important, although the teachers argued that they

report to parents mainly in order to justify their grading system. The findings of this study then reiterate that assessment is performed especially in order to meet accountability requirements.

Further emphasising the extreme accountability purpose, teachers in Sikka, Nath and Cohen’s (2007) research, for example, expressed frustration and concerns with the pressure they would have to endure as assessment was ‘mandated by the high stakes assessment

requirement of schools’ and their school administrators emphasized standardized test practice and a specific format and held the teachers accountable for student performance’ (p. 249). This seems to have led to the decrease in teacher morale and the widespread instances of teachers leaving their profession.

This section provided an overview of research on teachers’ beliefs and conceptions with respect to the purposes of assessment. I started by explaining Brown’ (2002) TCoA inventory and move on to integrate the same with Barnes, Fives and Dacey’s (2015) continuum of assessment purposes. Thereafter, I presented and reviewed the relevant studies by organizing their findings along the suggested continuum. I believe that there seems to be a consensus among most participants in these various studies wherein they perceive assessment as an important tool in education, regardless of the purposes they believe it serves.

Although these studies have been carried out in different educational contexts (tertiary and/or otherwise), there is a lack of pertinent studies which adequately cover Saudi higher-

educational contexts. Therefore, this study includes a new context – Saudi Arabia – and examines the beliefs of EFL teachers in four higher-educational institutions in Riyadh.

2.4.5.2 Beliefs about methods of assessment.

Research on teachers’ conceptions about the use of various assessment methods suggests a degree of agreement among the participants regarding the most desirable and dependable types of assessment (Büyükkarcı, 2014; Elshawa, Abdullah, & Rashid, 2017; Karim, 2015; Pereira & Flores, 2017; Thomas, 2012).

Elshawa et al. (2017) report the attitudes of 83 English language instructors from six universities in Malaysia on various aspects of assessment methods. In contrast with their treatment of assessment purposes, which I reported in the previous section, in this case they asked questions which did not elicit how much a teacher claimed to use different kinds of assessment, but rather, in general terms, indicated how they evaluated the usefulness of different methods, e.g. ‘The language skill can be assessed through true-false items’ (p. 46) and ‘Self-assessment by the student is a good method of assessment’ (p. 44). Here, then, they were accessing teachers’ beliefs in light of espoused theory (2.3.1.1.1). The participants reported that many types of assessment tasks were suitable for assessing every language skill they teach, namely reading, writing, listening and speaking. The results showed that the teachers regarded traditional tasks as more capable of serving assessment purposes than alternative ones. For example, the teachers rated multiple-choice items, true-false choices, essay writing and oral presentation as higher than self-assessment, student portfolios, oral interviews and peer assessment. This suggests that teachers’ beliefs are more conservative than the purposes for which they claim to use assessments – towards the extreme pedagogical end of the assessment continuum – while the methods they endorse are steered more towards the centre of the continuum.

Karim (2015) used an open-ended questionnaire to determine 25 Kurdish ESL teachers’ beliefs about item formats and classroom assessment procedures. The results indicated that all the teachers believed formal assessments (e.g. written exams or oral presentations) provided better evaluation than informal assessments (e.g. incidental, unplanned comments and responses). Also, it was found that 100% of the participant teachers used paper-and- pencil assessments as the primary format to grade students as per the tasks assigned. These findings suggest a preference for summative assessment, but the researcher attributed this to the anxiety and difficulty that teachers encounter while implementing formative assessment practices.

In Pakistan, Thomas (2012) compared the beliefs about selection of assessment strategies by untrained teachers (those who had never received any regular educational training) with those of the trained ones. The research located a pattern of similarities in their responses, indicating the teachers’ enthusiasm to use student-centred assessment strategies, although they would refrain from doing so as school policy required the use of summative assessment.

2.4.5.3 Alignment between beliefs and practice.

According to Borg (2003), effective classroom instruction takes place when teachers’ beliefs are congruent with their instructional practices. However, a study of the mainstream literature on teachers’ beliefs and their instructional practices reveals gaps and contradictions between what teachers say, think or believe and what they do. In this section, I discuss whether or not teachers’ beliefs about assessment exhibit any consistencies with their assessment practices. A few studies have addressed this issue, with all of them concluding that there were

practices (Büyükkarcı, 2014; Chew & Lee, 2008; Davis & Neitzel, 2011; Karp & Woods, 2008).

Büyükkarcı (2014), for example, investigated the formative assessment perceptions and classroom practices of 69 primary English language teachers. Using both qualitative and quantitative data collection tools, the researcher found differences between the teachers’ perceptions of formative classroom assessment and their real assessment practices. The participant teachers were found to garner positive beliefs and attitudes towards formative assessment. However, within classrooms, the teachers reported that they mostly used assessment methods for summative purposes (e.g. exam papers). Büyükkarcı (2014)

concluded that such a gap between teachers’ beliefs and practices exists because the classes are crowded, the teachers have heavy teaching loads and the amount of time spent in classrooms is limited.

Moreover, a similar mismatch between teachers’ beliefs about and practices of assessment was found in Davis and Neitzel's (2011) study. Through classroom observations and semi- structured interviews, the researchers examined 15 teachers in US middle schools to identify their conceptions of assessment and how these conceptions were reflected in their daily assessment practices. In general, the researchers found that the teachers’ conceptions of assessment and their assessment practices were incongruent:

These mismatches between the tenets of SRL2 and teachers’ assessment practices do not reflect deficiencies in teachers’ understandings of assessment. Instead, these mismatches occur because teachers are asked to satisfy various assessment audiences, and the interests of these audiences (p. 212).

In contrast, Chew and Lee (2008) developed an online questionnaire to measure teachers’ beliefs and practices concerning traditional classroom assessment. Descriptive statistics were reported together with analyses such as paired t-tests, ANOVA, factor analysis and stepwise regression were carried out on a sample of 148 facilitators3 from a polytechnic in Singapore. The analyses of participants’ responses led to the identification of three dimensions of