COMITÉ INTERNACIONAL DE LA CRUZ ROJA (CICR)

The case study was the most potentially vulnerable phase of the research, as it pertained directly to assessment outcomes for the student teacher. Participants were informed in the initial consent process that they could ask for the recording to be ceased, and for me to leave the meeting, at any point. This right was reiterated at the start of each assessment triadic meeting, and again at the outset of the interviews. Initial selection of the case study also asked institutions to select students who were not considered at risk of failing, in order to protect potentially vulnerable students. I also exercised my own judgement as a researcher and teacher educator to evaluate risk throughout the data collection of this phase.

3.10. Summary

This chapter has described the methodological framework within which the study has been conducted. The epistemological beliefs central to the study have been articulated, and the rationale for the methodological and design decisions provided. Table 3.2 is provided to offer a visual summary of the multiple phases of the research, and the specific design elements relevant to each stage, to serve as a quick reference guide. The following three chapters present the findings of the study. As the preface to each of these chapters, a brief description of the sample, data collection and data analysis procedures specific to each phase of the study have been provided, to set the context for the results which follow.

Table 3.2

Overview of the research design Phase One

Key Informant Interviews Phase Two Questionnaire Phase Three Case Studies

Timeline August- October 2011 October 2011 – May 2012 July – September 2012

Focus Institutional practices and

policy

Individual perceptions and experiences of student teachers, associate Teachers and teacher Educators Actual experience of practicum assessment, including the relationships between practicum participants. Key Research Objectives

To critically examine how a representative sample of New Zealand teacher education providers assess student practicum.

To critically examine the beliefs, perceptions and experiences of the key stakeholders in the practicum experience at five institutions representing a range of approaches to teacher education and practicum in New Zealand.

To identify the factors which create authentic assessment of students’ teaching practice during practicum.

Research Methods

Guided topic interviews, face to face. Participants given questions prior to interview Questionnaire/Survey – to all participants of

practicum within the institutions

Online survey using Survey Monkey.

Mix of closed and open questions. Some rating scales and belief assessments.

Case studies – including: guided interviews (post- assessment meeting), recording of assessment meeting, and analysis of documentation.

Participants Key informant at four

representative institutions, chosen on the basis of the different approaches and delivery of teacher education.

All students within the three year early childhood initial teacher education programme.

All teacher educators within the relevant programme who are involved in practicum assessment.

All associate teachers who support student in the relevant ITE programme.

One case study from each institution, including a student teacher in the third year of their study, their teacher educator and associate teacher.

Analysis of Data Foregrounding of institutional plane. Focus on analysis of institution policies and practices within the institution.

Qualitative thematic analysis, addressing key topics

Foregrounding of the personal/intrapersonal plane. Focus on analysis of individual experience. Qualitative thematic analysis of open-ended questions. Descriptive statistical analysis of quantitative questions Foregrounding of the interpersonal plane – considering the way in which the assessment is enacted in the context of relationships.

Qualitative thematic analysis, addressing key topics

Role of the researcher

Interviewer – personal but formal contact with participants. Developer of potential interview topics. Informed outsider.

Developer of

questionnaire. Distant contact with participants through instrument. Informed outsider.

Interviewer/observer. Personal but formal contact with participants. Developer of potential interview topics. Informed outsider.

Justification It is not possible to fully

understand a given situation without understanding the social and contextual factors that govern the way in which practices are manifest in a given institution/community (Rogoff, 2003). Focus on illumination and exploration, not comparison. Descriptive intent. Interviews selected as a means of gathering rich data, and allowing for clarification and expansion of data (Cohen, et al., 2002),

Recognises the importance of the story of the

individual, but in choosing a large scale survey seeks as many stories as possible to allow some inferences to be drawn. May gather less rich data due to the physical constraints of a questionnaire (Cohen, et al., 2002).

Important to gain an illustration of actual practise as it happens and to see if this matches self- reports and reflections – how these might be coloured, cleaned up perhaps? Gathers data in the moment. Rich data collection using multiple sources to flesh it out (Cohen, et al., 2002). Most complete picture on a macro-level. Allows examination of the dynamic relationship between participants (Tashakkori & Teddlie, 2003).

Ethical Considerations Anonymity/ Confidentiality Data usage Risk to participants Informed Consent Conflict of interest

Institutions made aware that details of the programme practicum component may give an indication as to the identity, but that in presenting results effort is made to minimise identifiable features. Pseudonyms used throughout. Data used only for disclosed purpose; the completion of PhD requirements and subsequent publications Minimal risk is apparent. Participants had control over disclosure, to minimise risk of commercial sensitivity concerns Full disclosure to participants of all components and anticipated outcomes of the study No evident conflict of interest. No professional roles in relation to the programmes involved

Questionnaires are anonymous. No identifying features collected. Results grouped according to institutional data sets for analysis purposes.

No apparent risk due to anonymity.

Identity of triadic members not disclosed. Pseudonyms used throughout. No risk to participants as I had no position of influence in the institutions involved, or direct involvement in assessment outcome 106

Chapter Four -

The Institutional Plane

The assessment of practicum takes place within the parameters of institutional policy and practice. While it is enacted and experienced by individual participants, the institution provides the regulatory system that shapes the practicum experience in both explicit and tacit ways. This chapter presents the findings of Phase One of the study, with a focus on the institutional plane. Findings directly address research objective one and three:

1. to critically examine how a representative sample of New Zealand initial teacher education institutions assess early childhood practicum; and

3. to identify the factors that support authentic assessment of student teachers’ practice during practicum.

This chapter begins with an overview of the sample, data collection and data analysis

procedures, and then presents the results of the interviews with key informants in each of the institutions, according to key themes.

4.2.Description of methods

4.2.1.Sample

Each of the four representative institutions was asked to nominate a ‘key informant’; a person who would have the knowledge and experience to be able to speak on behalf of the institution in relation to the policies and practices surrounding the assessment of practicum. Each

institution had a unique organisational and leadership structure, therefore the request for a key informant did not stipulate a specific role, but requested to interview the person within the three-year early childhood initial teacher education degree who would have the

knowledge to address questions related to the policy and practices of practicum assessment in this programme.

The final sample included a total of six participants, as two institutions requested that two informants be interviewed, to address different areas of knowledge or expertise. The informants held different positions within the institutions, although all held a position of leadership, either in the overall ITE programme, or in practicum. Once identified by the institution, each potential key informant was provided with an information sheet that described the study, the nature of their participation, and their rights as participants (see Appendix D). Written consent was received from each informant prior to data collection (see Appendix G).

4.2.2.Data collection

The method used for this phase of the study was a qualitative interview, using a general interview guide approach in which topics were pre-prepared, but the order and nature of questions was determined during the course of the interview (See Appendix I). The purpose of this phase of the data collection was to gain an understanding of the institutional context in which practicum assessment occurred. The intention was to understand the practices and processes that guided the assessment process, as well as the philosophy and policy that underpinned these actions.

The interviews were conducted between August and October, 2011. A suitable time and place for the interview was negotiated with each key informant. Each interview was held at the informant’s place of work, as per their request. The key informants were provided with a copy of potential interview topics (see Appendix I) prior to the interview, to allow time for reflection as well as to source additional information that may have been needed. These topics were generated within the framework provided by the research objectives, and were informed by the literature review and my own knowledge and experience.

Each interview took approximately one, to one and half hours. Effort was made to keep the interviews within the intended timeframes in awareness of the workload and other

commitments of participants. At times this required a decision to conclude an interview, even when there were further potential areas of discussion. More time with the key informants may have yielded greater information regarding the rationale for assessment practices; however it was necessary to balance the needs of the study with the needs of participants.

Each interview began with an explanation of the study and the opportunity for the key informants to ask any questions they had. Their consent to be interviewed and recorded was confirmed, and the right to cease the interview at any point reiterated. With the permission of participants, all interviews were digitally audio-recorded, and subsequently professionally transcribed. The transcription process raised an unanticipated issue, when the transcriber had difficulty interpreting some of the terminology used. Each interview was therefore listened to and corrected to ensure accuracy. This proved valuable, in increasing familiarity with the raw data which supported subsequent analysis. Transcriptions were then returned to participants for review. They were informed that this was an opportunity to check for accuracy, as well as to offer any clarification that they felt was needed. Each key informant approved their transcript and signed a transcript release (see Appendix J).

4.2.3.Data analysis

The analysis of the key informant interviews commenced with listening to audio tracks of the interviews on two occasions to refresh memory of the content. The transcript was re-read and the process of manual coding within NVIVO 10 began. At this point thematic categorisation (noding) was started. Coding began with a preliminary list of themes that had been drawn from the literature review and from consideration of the research objectives. An iterative

process was used, whereby quotes from the transcript were coded to the existing themes, and more themes were added as they were identified in the results.

When this initial process was completed, a more fine-grain analysis was pursued. Each of the broad themes was examined to determine if there were sub-themes. This required additional coding, and enabled more of the detail in the data to be explored. The final overview of

emerging themes is presented in Table 3.1, (p. 96). At the completion of this initial analysis, the generation of themes had been a positive outcome, but it was challenging to gain an overview across the institutions. At this point I chose to move to manual analysis that supported me to work with the data set in a more cohesive way. I took a large piece of paper and developed a matrix that allowed me to plot the responses from each key informant according to theme. This enabled analysis of how a specific theme was addressed in each institution and supported identification of similarity and difference. Each key informant transcript was reviewed three times to identify responses to key themes.

The use of a guided interview approach raised some challenges in the analysis of this dataset. Although the key informants addressed the same broad topics, specific areas within these topics may not have been addressed by everyone. This meant that it was not always possible to make ready comparison across institutions, due to the way questions were asked and answered. Where possible, in describing the analysis of results, similar responses across institutions are noted, as well as findings that are unique to each individual setting.

The following discussion presents the results of the key informant interviews. The

characteristics of practicum in each of the four settings are outlined, and then findings are presented according to key identified themes.

4.2.4.Presentation of findings

In presenting the findings of this chapter, the findings of the interviews have been collated across the four institutions, and discussed in relation to key themes of the study, in particular assessment philosophy and principles, assessment practices, assessment challenges and authentic assessment. Quotes have been integrated throughout the presentation of results to illustrate the participant’s narratives and to illuminate key points. Coding has been used throughout the results to identify the source of the quotes and support an audit trail. Coding adopts the following format: (Institution Pseudonym/Role of Participant/ID of participant/Page number: line numbers), e.g. (TAHI/KI/1/5:3-7). In this phase, the abbreviation ‘KI’ is used to denote the Key Informant role.