In qualitative research, the data collected are usually “a large, cumbersome
database because of its reliance on prose in the form of such media as field notes, interview transcripts, or documents” (Bryman 2008, p. 538). This huge amount of data can sometimes result in the researcher avoiding a thorough and appropriate analysis. Merriam (1998, p.178) sets the principles of data analysis as following:
Data analysis is the process of making sense of the data. Making sense of the data involves consolidating, reducing, and interpreting what people have said and what the researcher has seen and read -it is the process of making meaning.
As this is qualitative, interpretive research, a thematic analysis approach was adopted for this study. Although this approach has been criticised of being only a tool or skill used across a range of qualitative methods (Boyatzis, 1998; and Ryan & Bernard, 2000), in the last ten years this approach has been “recognised as a qualitative research method in its own right” (Willig, 2008, p.179). Fereday and Muir-Cochrane (2006, p.68) explained that a thematic analysis approach entitles “a search for themes that emerge as being important to the description of the
phenomenon [under investigation]”. As this approach to data analysis is not restricted to a particular theoretical approach to qualitative research, I needed to identify what exactly the themes in my analysis to represent (Willig, 2008). This representation was guided by my two research questions in addition to the epistemological point of view of the study. However, Taylor and Bogdan (1998, p.140) elaborate further by stating that “data analysis in qualitative research is a different partas it requires thinking and reasoning, rather than mechanical and technical process. It is an inductive process”.
Based on these premises, the initial process of data analysis started immediately after the pilot study. Indeed, the three pilot interviews were transcribed and during the interviews, although they were recorded, I took notes wrote comments about
anything I considered relevant. As explained previously in Section 4.5, the interviewees’ comments about the clarity of certain questions were taken into consideration and several items were slightly modified. This also led to dividing the interviews into two parts: the first part discussed how teachers view their role in the English language assessment, while the second part dealt with their beliefs
concerning language assessment in the institution.
Upon completing all interviews and returning back to the United Kingdom, I did my utmost to transcribe accurately all the recordings by listening to the interviews several times. I then read several times the transcripts and emailed them to each participant requesting them to read them carefully and send them back with any comments or corrections they might have. Most participants did not make any significant changes other than correcting minor spelling and grammar mistakes in the transcripts. Three participants added certain points, but did not actually change what they had said during the interview.
Upon analysing the data, each interview transcript was examined carefully for relationships between the different parts of the data. The interview transcripts were then coded and different categories were derived (Hsieh & Shannon, 2005). The first stage of the coding process was done on paper by highlighting different ideas that derived from the data. This led to developing different mind maps that enabled me to arrive at all the different themes and categories and make sense of the huge amount of data accumulated (See Appendix 4). Subsequently, the MAXQDA
software proved highly useful for the process of allocating each piece of the data or quote to its specific category. This programme also helped in organising the data as well as searching for specific themes:
Figure 2 General Overview of Coding Using MAXQDA 11
This tool also proved its utility because many of the quotes and codes could be entered under several different categories. The data were grouped under three major themes: teachers’ roles, teachers’ beliefs and challenges. The codes from the transcribed interview were put in a table to show how they cluster into
categories and how the categories became the final themes. A sample of one sub- theme is presented below (Table 3) with the full theme presented in Appendix 5:
Table 3 Sample of Data Analysis Stages
Stage 1 Stage 2 Stage 3 Stage 4
Quotes Codes Categories Sub Theme Theme
Document: 01 Wadee (M)
Position: 33 - 33
… of course not all teachers are involved in designing tests in the institution. Only the teachers work in the assessment committee have a saying in the test format and questions. You have to keep in mind that all of the questions have to be multiple-choice questions.
Document: 01 Wadee (M)
Position: 98 - 98
my role did change over the past few years. I remember that I was more involved in the assessment by preparing exams and actually choosing the way to assess my students. I mean in the past the teacher could teach we had a lot of power and flexibility…
Document: 03 Iqbal (M)
Position: 69 - 69
when it comes to designing the test I believe teachers have a small role in that. Tests are initially designed by the assessment committee and then given to us as teachers to administer them with the students.
Document: 04 Saeed (M)
Position: 254 - 257
as I told you before these tests are done by the assessment committee people. I heard that sometimes they would make teachers take the tests and then give them feedback and comments.
Document: 20 Afnan
Position: 16 - 16
teachers don’t have a role in preparing any of the tests
Non- involvement in test design History of teachers’ role in test design No role in test design The role is limited to the assessment committee No role in test design Test design and summative assessment Teachers Roles in summative assessment Teacher s Roles