A sociocultural perspective is connected with the social nature of knowledge which requires the researcher to access multiple sources of data within the context to gain an in depth understanding of the assessment practices. Data for this study were collected from multiple sources, including the analysis of observations of lecturers and students in their classes; interviews with lecturers and students; and written documents of EU. In the following sections, the use of these research methods is discussed.
3.2.3.1 Observation
This study used observation to collect information of how the lecturers use assessment to support learning, as observation is considered a powerful tool for understanding phenomena in particular contexts (Merriam, 1988; Simons, 2009). Within constructivist inquiry, researchers need to immerse themselves into the natural context to explore the phenomenon. The researcher visited the lecturers’ classrooms
to capture a comprehensive picture of their practices of assessment. The observation focused on collecting data of what and how lecturers and students used strategies of AfL. Observed information about the classroom context was noted to assist the investigation and analysis of the assessment practices [Appendix A]. Simons (2009) identifies that without knowledge of the context, the researcher may not clearly understand an issue.
In this study, non-participant observation was utilised. Observing without being involved in classroom activities helped the researcher gain a focused comprehensive picture of the assessment practices for learning of the three lecturers in EU. The researcher visited five sessions of Ly and Tung and four sessions of Hoa to observe their assessment practices. The observations focused on the following aspects of each session:
• AfL strategies as used by the lecturers;
• ways in which lecturers used the feedback, peer and self-assessment, questioning, sharing learning intentions and success criteria;
• the participation and reflection of students about the assessment activities; and
• the classroom atmosphere and the context of classes, such as number of students and the teaching resources.
Non-verbal cues such as gesture and facial expressions in classroom interactions were also important to note because these can provide additional information for understanding the phenomenon (Simons, 2009). Following is an excerpt of fieldnotes compiled in the third observed teaching session in Ly’s class. This part constitutes the general comments at the end of the lesson.
TRANSLATION OF THE EXCERPT
The teaching session was interesting with a lot of classroom activities. Students’ participation was generally active. The learning environment was democratic and safe. The teacher used many active teaching methods to stimulate students’ thinking and interest. For example, case study, role play and group discussion. The teacher encouraged her students to share and voice their own ideas. The technique “writing a learning diary” combined with the technique of “three and three” was expected to be a useful strategy to develop self-assessment and also gain feedback on teaching. The lecturer was planning to review the lesson and then adjust teaching accordingly. The teacher gave feedback (FB) to her students, suggesting many ways to enhance their Communication Skills
The teacher focused on eliciting students’ experience and created opportunities for them to learn by doing (co-construction), which encouraged more students to engage in their learning.
Parts of lecture were structured logically and clearly, usually by her use of questioning. She used a range of questions such as yes/no and wh-questions to stimulate student learning. She also used brainstorming strategies.
As the lesson occurred after a long public holiday (30/4), it impacted on student learning, i.e. attitude, tiredness. Also, they had to move to a new lecture hall as projector in the current room was broken and the technician came to repair it. The teacher had to use chalk and blackboard to write the main ideas of discussion or content of the lesson. It was very blurry so students at the back of the lecture hall could not see it (poor facilities - hindrance).
Place, 02/05/2013 Observer Signature
Apart from using field notes, the researcher asked lecturers for permission to record their lectures by video. To reflect the practice of AfL in classrooms, interactions between lecturer and students as well as among students in five teaching sessions were video-recorded. During the process of observation, the researcher remained non- judgemental about what occurred in the classrooms. Observations occurred five times to ensure that the data collection of assessment practices was comprehensive. This is one way to enhance trustworthiness. Information from the interview prior to each session and from the documents was linked with the observation process.
3.2.3.2 Interviews
Interview is considered a useful method of collecting data for understanding the phenomenon as perceived by the participant (Merriam, 1988; Stake, 1995). In this study, interviews were used to elicit experiences of both lecturers and students in assessment activities in their classrooms. The lecturers’ perceptions of learning and assessment were articulated and further probed through the use of interviews.
Semi-structured interviews with lecturers
This study used semi-structured interviews, a flexible and adaptable implementation of a set of predetermined questions, to collect data (Creswell, 2008; Merriam, 1988; Robson, 2002). A list of open-ended questions was prepared before each interview to investigate beliefs and experiences of the lecturers in their assessment practices. During the interview, apart from the predetermined questions, probes and follow-up questions were used to encourage respondents to provide a rich description of their assessment practices. Ly and Tung were involved in ten interviews, while Hoa participated in eight interviews before and after observed sessions in the unit that they were teaching.
Interviews prior to lectures
Interviews conducted before the lectures aimed to gain an understanding of the teaching and assessment strategies that the lecturers intended to use to support learning. The interview before the first lecture was the longest, approximately one hour in duration [Appendix B]. It was conducted to gain information on the learning intentions of the lecture, and also to understand the particular teaching approach that the lecturer adopted. It was assumed that their approaches to teaching may influence the preparation and the implementation of the particular lectures. The next four
interviews took around 15 to 20 minutes and included questions which focused on teaching and assessment strategies that lecturers intended to use in their lectures to promote learning; their justification for choosing these strategies; and the outcomes that they hoped to achieve with these strategies (Appendix C).
The interviews after lectures
Interviews were also conducted after each of the observed sessions for each lecturer. Interviews after the observations helped to delve into the perceptions and attitudes of the lecturers about student learning and assessment during a particular lecture; the ways the lecturers had been using assessment strategies in these classes; and the factors that supported or hindered lecturers in their use of assessment strategies. The interviews took about 30 to 45 minutes and involved questions that related to the observed assessment practices. Other questions depended on what had been observed, as well as probing for further details [Appendix C].
Focus group interviews with students
From a sociocultural perspective, it is also very important to gather information from students to gauge the impact of assessment strategies on their learning. Therefore, the study employed focus group interviews in which the researcher sought information from small groups of students. This research technique was used because it is “economical” (Simons, 2009, p. 49) and could provide a sense of the agreement level among students concerning current assessment practices.
Focus groups were conducted with seven students in Ly’s class and six students in Hoa and Tung’s classes after each of the five lectures. Each group interview was 45 minutes to one hour in duration, and consisted of students who had achieved high and low assessment results in their previous semester. Each group had both male and female students, except for the student group in Tung’s class, which was all female, reflecting a gender preference for teacher training and social science. This purposeful selection aimed to discover the range of experiences of different students with the same teaching and assessment activities.
The first group interview in each class lasted around one hour in order to collect data on students’ experiences with assessment strategies in that lecture, as well as EU’s assessment practices [Appendix D]. Some follow-up or probing questions were raised in the group interviews relating to data collected from the specific classroom
observations. Further, questions to facilitate group discussions were also conducted to gain a rich picture of the students’ experiences with the assessment strategies [Appendix E].
Simons (2009) identifies issues with group interviews related to students’ responses, including responses being either too convergent or too divergent. Another issue is the dominant voice of one member in the group while others tend to be quiet (Simons, 2009). These issues were also seen in focus group interviews of this study. To deal with these unintended phenomena, the researcher attempted to create a comfortable environment for dialogue and asked questions evenly of each student in the group. In this way, the researcher ensured that every student had an opportunity to respond to the questions. Moreover, prompt questions were asked of students when they had different experiences and opinions about the same assessment strategies. These questions included:
In terms of ....A said that, what about your experience of this, B? Do you agree with him/her or do you have different ideas? And why do you think that? In the lecture, I saw this situation...Could you tell more about...?
How did you feel when the lecturer...?
For interviews, predetermined questions play an important role in eliciting information from participants. To ensure academic standard interviews, questions for lecturers and students were devised and then trialled with one lecturer and one fourth- year student in EU via Skype. This process is detailed in section 3.2.4.
Apart from refining predetermined questions, a range of considerations were made in the conduct of the interviews. First, interviews were conducted in a private area on the University campus to facilitate open communication. As interview involves the exchange of information through language, the effectiveness of an interview depends on the interactions between the interviewer and interviewees. Ideally, the researcher should be able to actively listen to respondents (Merriam, 1988; Robson, 2002; Simons, 2009). Robson (2002) suggests that regular reflection or rephrasing of what interviewees are saying is necessary. Moreover, being EUtral and nonjudgmental of interviewees’ responses is also suggested by Simons (2009). Non-verbal cues such as gesture and facial expressions were noted during the interviews, since these may add meaning or clarify the participants’ words. Further, consciousness of these distractions or prompts is necessary in research to avoid collecting unreliable data.
Along with the support of audio-recordings in interviews, the researcher combined both observation and note taking when required.
3.2.3.3 Documents
In this qualitative inquiry, documents were a useful source of data (Creswell, 2008; Merriam, 1988; Simons, 2009) for revealing the practice of AfL in EU. Documents are often understood as written documents, containing “a ready-made source of data” (Merriam, 1988, p. 104). Along with interviews and observations, the analysis of documents is known as a common research technique in case study design. According to Simons (2009, p. 64), the analysis of documents provides “a helpful precursor to observing and interviewing”. For example, information on assessment policy in Vietnamese higher education was used to develop questions in interviews, and to frame intentions compared with the lecturers’ enactment of these in their teaching practices.
There are many types of documents which may provide relevant information to explain a phenomenon. Although data in documents can be a useful source for research, careful selection of documents is necessary. Merriam (1988) highlighted that documents are not usually created for research purposes and therefore their accuracy needs to be carefully considered. For this study, data from the formal documents included MoET’s assessment policies in higher education, the University’s annual reports, vision statements, rules and regulations, and examination results. Informal documents gained from newspapers, and EU’s website were analysed. The study also used information collected from other documents such as lecturers’ lesson plans and students’ work. These documents helped the researcher to explore the following:
• how these policies shaped the assessment practices in this University;
• if the lecturers’ assessment practices aligned with the policies of this University and MoET;
• what individual, social and cultural factors may have created synergy or tensions between assessment practices and the policy;
• the relationship between the intent, the performance, and the outcomes of assessment activities in the classrooms; and
In conclusion, interviews, observations, and documents were employed to collect data in this study. The following table summarises the participation in the
research of the three lecturers and 19 students.
Table 3.3
The Participation of the Three Lecturers and Students in the Research
Research Methods Case One
Tran, Thi Ly
Case Two
Duong, Thi Hoa
Case Three
Nguyen, Van Tung
Observation 5 teaching sessions 4 teaching sessions 5 teaching sessions Interviews with
lecturers
10 (5 prior – 5 post) 8 (4 prior – 4 post) 10 (5 prior – 5 post)
Focus groups interviews with students
5 post each observed teaching session
4 post each observed teaching session
5 post each observed teaching session Documents MoET and EU’s assessment policies, annual reports, and websites
Lecturer’s lesson plans and students’ work