SUPERFICIE SEMBRADA DE ARAZA EN ZONAS DEL ECUADOR

Student perceptions give important insight into their learning experiences (Parlett & Hamilton, 1977). The objective of this phase was to explore the factors that have influenced the

development of CT for a diverse sample of students (the outcome of Phase I) who are about to graduate and who have shown potentially good CT. The outcome was a set of possible factors internal and external to AUC that have had a positive impact on these students’ CT development. This was done in three steps:

1. A written pre-interview questionnaire was sent via email to students, which they returned to me prior to the actual interview. The questionnaire (see appendix A-1) to confirm students’ majors, and to know more about students’ family background, travel experience,

extracurricular activities, and favourite courses. This was done to help me understand each students’ background further, and to help me ask additional questions to prompt students. It is expected that students' pre-university backgrounds and experiences not only affect their incoming CT, but influence their pathways through university (Terenzini et al., 1995). 2. Individual interviews with students chosen from Phase I; these interviews were semi-

structured, but scaffolded using a detailed definition of CT (see appendix A-2 for the main questions), as it is not straightforward to reflect on influential factors in a one-hour interview, and I felt it would be difficult to reflect on CT as an abstract concept in a holistic manner without breaking it down into something more manageable; hoping not be patronizing, I still felt the need to consider the maturity of the students in order not to overestimate their reflective abilities.

3. After completing the interviews, and while conducting my analysis, I told students I might contact them again, and I sometimes sent email questions to individuals clarifying certain aspects.

4.8.2.2 Methodological Issues and Positionality

While interviewing students in Phase II, my previous interaction with them was both an advantage (already established rapport, shared experiences from the course) and a disadvantage (e.g. power relations – I used to be their TA and responsible for their grades; they may be automatically

influenced to think of the Scientific Thinking course, they may skip over certain details knowing that I implicitly understand them). I sometimes explicitly asked questions directly relevant to the course when I felt it would help carry the discussion deeper (e.g. I sometimes referred to certain things they wrote and say something like “that displays great meta-cognition, how do you think you developed this?”).

As I mentioned before, I expected scaffolding to be important, and that I would need to find ways to help them dig deeper into their thoughts, without leading them to particular responses. In the end, some interviewees were able to reflect and express themselves deeply without much prompting from me, providing rich data, while others required more scaffolding and prompting, and had difficulty identifying more than a few influences on their CT. While this is not unexpected, I recognize that this means the students more “eloquent” in the interviews had their experiences more deeply explored, and their interview data ended up being more prominent in the thesis. Similarly, students who had particularly interesting or deep experience influencing (and in some cases, hindering) their CT have ended up taking up more space than others in the analysis. 4.8.2.3 Ethical Issues

Informed consent was taken – my interviewees were presented with a briefing on the research and asked to sign the standard University of Sheffield forms.

I protected subjects’ identities by anonymizing all data collected and presented about them, and storing it in password-protected media. The only issue with this aspect is that the instructor of the course knows the students very well and might be able to identify individuals even without their names.

4.8.2.4 Implementation

4.8.2.4.1 Pilot Interviews

Prior to conducting the student interviews, I piloted one possible interview question (about questioning authority) with some AUC alumni via facebook (asynchronously). The purpose of this “pilot” was not to use the response data in the thesis, but to find out how the question was answered, and to get feedback on wording the question.

Responses received showed progress throughout university, and a variety of factors influencing this. However, the majority of alumni cited factors outside of university and some could not remember specifics related to university. This supported my decision to interview undergraduates instead of alumni. The quick responses of alumni indicated it was not that difficult for them to assess the change in their “questioning” level and come up with examples of factors affecting that change. Most replies were actually quite detailed. While alumni's maturity might contribute to more reflexivity than undergraduates, it was an indication that such reflection was possible. I also conducted two complete interviews with non-AUCians, one face-to-face, and one on the phone, to test out my interview questions. I discovered that including Perry’s model as an explicit question was too complicated, and I removed this question.

4.8.2.5 Student Interviews

Due to personal circumstances, interviews were conducted online on audio via Skype or MSN. I have reflected on this in the methodological challenges below.

All interview data was placed in an MS Access database I had designed myself. Student interviews were transcribed and coded using a grounded approach, where categories of “themes” emerged from student interviews. For each “influential factor” (hereafter referred to as factor) coded, I kept track of which question the student was answering, which led me to which aspect of CT it helped develop, so that I could then easily search the database for e.g. which aspects of CT a certain course helped develop; or which students mentioned travel experiences as influencing their CT. This “coding” was done in several iterations to tweak the categories into levels of detail.

4.8.3 Phase III

A job of the educational researcher is to begin examining the work of effective teachers, teachers working intelligently with appealing content, in an attempt to validate

generalizations that may be made about the elements which make the teaching strong. (Atkins, 1977, p. 80)

Teachers’ beliefs in their personal efficacy to motivate and promote learning affect the types of learning environments they create and the level of academic progress their students achieve. (Bandura, 1993, p.117 quoted in Tsui 2001)

4.8.3.1 Description

The objective of this phase was to further understand how internal AUC factors have influenced students’ development of CT. This was done by choosing the most prominent factors emerging from phase II and conducting one of the following for each:

1. Interviews (interview guide in appendix A-3) with instructors (in courses) and administrators (in university-wide strategies or activities) about “how” they think they are helping students develop CT, what motivates them to do so, and how they tackle issues of access to students. The students will have already elaborated on this a little bit, but the interview aims to provide a more practical guide of what is done, and how others can replicate this “good practice”; these interviews were loosely-structured, with some questions common among them, others designed specifically for the person being interviewed (e.g. based on something the student said, or some other aspect I wished to interrogate), and often involved many unprepared questions as the conversation flowed.

2. Observations (of classes, university events, extra-curricular activities in action) to understand how they are conducted and how they influence CT

The final choices of whom to interview resulted in at least one administrator and at least one instructor interviewed for each of the emergent themes in the findings chapters. This, in addition to other documentation used, served to ensure a level of triangulation for each of the themes, such that I could cover the theme from the viewpoints of various participants.

4.8.3.2 Methodological Issues and Positionality

I expected interviewing and/or observing instructors or AUC administrators in phase III to be the most “sensitive” part of the research, as some people could feel uncomfortable opening up to someone who is so much a part of their daily lives. Also, they were likely to feel that a lot of what they needed to tell me was implicit in what they say and I would have had to ask them to be more explicit. At the same time, I have a good working relationship with many people in the AUC community and I am often invited to help faculty assess their own teaching via observation or interviews/surveys with students. And since in this research I was contacting them as “exemplars” in developing CT, they were unlikely to feel threatened or judged.

One issue is response bias. Individuals who know me personally are more likely to respond quickly to my request for interview, and more likely to be flexible in terms of meeting time, and to be open in the interview based on established rapport.

Another important issue to keep in mind is my own bias about each instructor based on my previous knowledge of them. For example, I observed several classes taught by an instructor who had given a workshop related to CT31, so I was expecting a wonderful class. However, I was

extremely disappointed by how didactic her teaching style was and how she seemed to discourage divergent discussion and expression of dissenting views. Having observed this, I was surprised that her students were actually very pleased with her teaching style. Upon reflection, it is possible that I had expected “too much” of her, and that without my pre-conceptions I might have judged her differently. Such bias is unavoidable when one knows research participants deeply. Instead, I have tried to use my previous knowledge of participants to the benefit of the research by referring to previous interactions during the interview, thus enriching the interview. This takes advantage of my long engagement with them to enhance the credibility of the research, relying on more than merely the interview "moment".

One other issue I was expecting to face was that administrators and faculty who are aware of the value of CT might say things that they think are “correct” but that they do not necessarily do. One way to overcome this was to ask them questions to elaborate specific aspects that students had mentioned in phase II, and then allow them to say what they would like to do more/better if they could.

4.8.3.3 Ethical Issues

Informed consent was obtained from participants in written form for all interviews. No specific consent was obtained for ethnographic observations, but approval from AUC to conduct research on campus was obtained mid-way through my research32. AUC only recently started implementing a policy of Institutional Review Board (IRB) approval. The problem with the IRB approval process is that it is based on health-related research and emphasizes potential physical harm to subjects, and

31

Upon reflection, I think she meant “critical” in the Freirian way, wanting students to rebel against the power of corporations, but implemented it didactically (pushing students to criticize corporate fraud), rather than “critical” in the North American way I had thought she meant (allowing students to see a different view from hers)

32

review boards expect to be able to see survey questions, for example. This renders the IRB approval process problematic for qualitative research, where the ethical issues are more subtle (e.g. related to protecting anonymity), where potential harm is less obvious, and where the research methods are not pre-defined in advance (Lincoln, 2005; Morse, Niehaus, Varnhagen, Austin, & McIntosh, 2009).

I attempted to ensure anonymity in all data collection materials and final writing. Where individual faculty/administrators refer in their interviews to something that would easily be traced back to them, I tried to remove that detail, unless they specifically give me permission to keep it.

However, some people’s positions (especially administrators) are likely to be identifiable when an informed AUC insider reads the thesis. I have informed participants that this may be the case. All instructors/administrators were shown the interview transcript for a quick review in case they felt I had misunderstood/misquoted them. Thankfully, none requested changes. However, one administrator refused to have his interview recorded on audio, and preferred that I take notes only33.

4.8.3.4 Implementation

The full list of six administrators and eight instructors interviewed is provided in appendix B. I decided not to interview and observe the same individuals to avoid making them feel that I am evaluating them – interviews highlight their understanding of how they teach CT, whereas observations would have focused on my own interpretation of how they teach CT.

Observation would have allowed me to see things that seem to influence CT even when the instructor had not intended this (e.g. Eisner, 1977); not surprisingly, other research (Gladwell, 2005) shows that people often have difficulty putting their perceptions/intentions into words, and this is where observation is particularly helpful. However, when I contacted instructors, I gave them the choice between granting me an interview, and allowing me to observe a few of their classes. Many were open to either and even both options, but because of time and scheduling constraints, interviews were deemed more efficient in getting an overview and insight into

instructors’ motivations, and thinking processes, which can only be gained by asking them directly.

33

His reasoning for this was that he did not wish to be misquoted. I found this strange, but did as he wished and chose not to waste time arguing.

Moreover, my analysis tended not to focus on the specifics of pedagogy, but on aspects of the instructors' intentionality in developing CT, which would not necessarily have been clear in observations. It is possible that, had I conducted observations, there would have been more pedagogical detail in my thesis.

In hindsight, given the time limitations, interviews were more appropriate to conduct than class observations. Observing a small number of classes would not necessarily have been representative enough of each instructor’s teaching: scheduling and logistical issues would have forced me to choose between either observing a handful of instructors frequently, or a larger number of instructors infrequently. Besides, the focus of my analysis is on curriculum rather than pedagogy, and so class observations would have shifted the focus to a more micro-level than the rest of the thesis.

As a partial solution to the interview/observation dichotomy, most of the instructors I interviewed are ones I had observed previously either in a teaching setting, or giving presentations. This had a dual advantage: it helped me decide to choose them to be interviewed, as I had more data than was mentioned in student interviews; and it provided more material for discussion during the interviews as I could refer to those times when I had observed them previously. It also satisfied the quality measure of "prolonged engagement" proposed by Lincoln & Guba (1986/2007). Although my previous knowledge of these instructors may be considered to have biased me towards them, this is not particularly problematic here, since the purpose was to find “exemplars” who teach CT, and this is how I approached the interviews. There was, however, a response bias in the sense that instructors who knew me personally responded to my emails faster and were more flexible in setting up interview appointments.

As I started writing up results and discussions, I found areas that I wished to explore further, and I supplemented my data with:

1. Observations (of university events, extra-curricular activities in action) to understand how they are conducted and how they influence CT

2. Reflections on my own personal experiences with some of these factors, where I had appropriate depth of experience. Examples include Soliya (chapter nine) and MUN (chapter eight)

3. Analysis of additional university documents (e.g. Institutional Research surveys, AUC catalogs) and research (some of which was conducted by me with other AUC faculty for different

purposes, mainly for conference presentations) to provide further evidence where appropriate

4.9 Methodological Challenges

Before describing the research design in detail and discussing the implementation, I share some methodological challenges that influenced the implementation.

4.9.1 Changes to Personal Circumstances

Even though for the most part, I was immersed in the AUC context throughout this thesis, there were several interruptions that affected my research. First, I lived in the US during 2007-2008, a time at which I had to conduct student interviews. After discussion with my supervisor at the time, I opted to conduct these interviews online via audio on Skype or MSN34, similar to a telephone interview.

Literature on qualitative telephone interviews is rare (Sturges & Hanrahan, 2004; Novick, 2008), as telephone interviews have been more often used for structured survey research, where

participants and researcher are strangers (Cohen et al., 2000), whereas in my case, I was interviewing students I had semester-long interaction with, and there was already rapport established. Telephone interviews are sometimes viewed with scepticism because of the contextual losses from lack of visual cues (Sturges & Hanrahan, 2004; Novick, 2008), whereas there is no evidence that there is significant loss in quality of interviews as compared to face-to- face interviews (Novick 2004). Instead, there is evidence of no significant differences in quality of data collected (Sturges & Hanrahan, 2004; Kazmer & Xie, 2008), and that sometimes telephone interviews are even preferable as respondents can be more relaxed and open about sensitive topics (Novick, 2008), and may consider the telephone interview a more convenient option (Sturges & Hanrahan, 2008).

34

Getting student contact information was in itself a struggle. I was able to get some students’ emails from their Turnitin.com accounts, and was able to reach others through facebook. Thankfully, most of them responded openly.

My decision to conduct audio interviews could have risked compromising the rapport possible in a face-to-face interview35, but since I already had rapport with the students, and all of us were comfortable with this technology of our generation, for the most part, I felt the interview quality was not compromised – thankfully all students had access to a high-speed internet connection in a comfortable place from which to meet me given the 8-hour time difference. In one particular instance, a student did not have a high speed internet connection at home and had to talk to me from a friend's house, which may have affected her level of comfort. Because of time zone differences we ended up doing the interview over two days. Although these were different conditions than other interviews, this particular student talked about how doing the interview on a different day might have produced different answers – so in her case, she did have two separate days to do so.

Out of nineteen students contacted, seventeen initially responded, and I was able to interview thirteen. I did not pursue the remaining students (who did not respond as quickly to follow-up emails) because student interviews were becoming repetitive in terms of student demographic