EL REPORTE DEL DESASTRE

Any research is bound to have limitations. Best and Khan (1989, p.37) state that “limitations are those conditions beyond the control of the researcher that may place restrictions on the conclusions of the study and their application to other situations”. In this section, I indicate my awareness of the boundaries caused by time, the sensitive issue of assessment, and other uncontrolled circumstances. For instance, the fact I was a Saudi national and conducting my research in Saudi Arabia might have made my participants, who are mostly non-Saudis on yearly contracts, cautious and careful when answering my questions. I recognised this and assured them of the confidentiality of whatever they might say in the interviews and took measures to ensure that no harm will come to any of the participants in the study by using pseudonyms for example.

I should mention that interviews with the male teachers ran very smoothly and they were face-to-face. However, my interviews with the female teachers took place over the telephone due to cultural reasons. Transcribing the data from a telephone recording was not as easy as transcribing the data from a regular interview. I tried

my best to transcribe everything however some of the words were unclear in the recordings therefore they were not transcribed.

Another obvious limitation was that I only used one research method, semi- structured interviews, in collecting the data from my participants. It would have been really useful to have access to exams and other assessment-related

documents since according to Hopkins (2014), they could provide useful insights and understandings of issues that are not available from other sources. It should be noted that the ELI administration made it very clear to me that such documents wear considered very classified documents and access to them is limited only to faculty members involved in testing and assessment. Those documents could have helped me in gaining a better understanding of the assessment policies and

practices in the ELI and would have also highlighted the nature of the teachers’ role and involvement in assessment practices in the institution. In addition, adding a questionnaire might have provided more breadth and could have served to enhance the trustworthiness of my data. However, as my research was trying to reveal the teachers’ beliefs through interviews, I felt that using a questionnaire would have made me assume a pre-established set of beliefs of the teachers, which was not the objective of the research. Using only interviews made me understand these beliefs leading to more in-depth questions on their attitudes towards their role in assessment. Moreover, from previous personal experience, teachers in this institution tend to receive a lot of questionnaires and rarely give any attention to them. Finally, the use of questionnaires does not fit into the framework of the study but should, nonetheless, be recognised as a limitation.

4.11 Summary

In sum, this chapter explored the previously outlined context and the literature review to present a justification for the chosen qualitative approach. It also shed light on the tools used to collect the data while demonstrating ethical

considerations of the research. In addition, the data analysis process was

explained and examples provided. The following chapter will explore the findings of the study.

Chapter Five: Findings

This chapter presents a summary of findings that were derived from the data analysis process. The following Figure 4 shows the themes and categories from the data:

Figure 4 : Themes and categories derived from the data analysis

As the above figure shows, the first theme that derived from the data revolves around the role(s) of the ELI teachers in language assessment practices in the institution. The participants were asked about their understanding of and attitudes towards their roles in language assessment and their responses were classified –

Them

es

The Washback Effect

Administration of Tests Marking Teachers unable to provide Feedback Role(s) in formative Assessment Role in Design Administration of Tests Unbalanced Marking Feedback 2. Teachers' Beliefs

Top Down Policy Curriculum vs.

Assessment Teacher & Assessor as

One Assessment Committee members' Qualification

Stake Holders Needs Professional Development & Testing

3. Challenges

Time Constraints The Lack of Academic

English Lack of Student

Motivation Rewarding Students’

Attendance Tests are Too Easy Unreliable Placement

for data analysis purposes- into two broad categories according to the type of assessment, i.e. summative and continuous. The first category was further divided into five categories: design, teaching for the test, administration, marking, and feedback while the second category was divided into four categories: design, administration, marking, and feedback.

The second theme revolves around the teachers’ beliefs with respect to language assessment. Participants were asked about the possible reasons for their role –or absence of role– in assessment as well as their views and attitudes on their role in terms of satisfaction. The reported responses by the participants were again classified into five main categories: ELI policy, teachers’ qualifications, learners’ needs, issues of teacher freedom & power, and professional development.

Finally, the third emerging theme dealt with the possible challenges facing teachers with regards assessment. They were asked whether the current ELI assessment practices were the source of any particular problems for the teachers or their students. The responses reported by the participants provided several key challenges, including, but not limited to, amount of assessment, student attendance, access, washback, placement tests, time constraints, failure and suitability. Each theme and corresponding categories are analysed in greater detail in the following sections.