La danza del Tango en: in aeternum

Since the focus of this research was to explore, describe and interpret the approaches of individual students to learning, the learning cultures adopted by student teams and some general learning outcomes, an ethnographic methodology was used. This section provides a rationale for using an ethnographic approach and an explanation of the data collection and analysis methods used for this study.

My extended contact with the engineering community as an electronics and communication engineer and then as an engineering teacher at Victoria University prompted me to investigate student learning in higher education engineering settings. As Victoria University was currently transforming its undergraduate engineering courses from lecture-based education to problem-based learning, it provided a suitable location for conducting this research study.

Undergraduate engineering education involves a gradual transformation from being a student in the engineering classrooms to a fully-fledged engineer. I believe that realities of such a transformation are situated in the context and cannot be isolated

62 from the context. Moreover, literature suggests that learning acquired through PBL is a classic example of situated learning (Brown et al., 1989). This recognises that student learning in a PBL setting involves more than merely being taught, but is an active collaborative process situated in communities of practice (Lave & Wenger, 1990). The theory of situated learning underpins a lot of thinking concerning collaborative learning in teams and professional development, which in turn informs the way we think about student learning in PBL teams.

Mixed methods and qualitative methodologies are most prominent among researchers who focus on situational perspectives in higher education. Research takes place in situ and reflects on assumptions that knowledge is constructed within specific contexts which have specific situational affordances; that is, the relationship between learners and the properties of specific environments.

While experts claim that quantitative methods are better than qualitative methods and vice-versa, it is only when we step back from these arguments and scratch beneath the surface, we come to realise the relative benefits of these methods. In qualitative studies, methods used are varied but the focus is often on the increased participation in specific communities of practice, the relationship between learners and the properties of specific environments and the distributed nature of knowing in specific communities. A major feature of quantitative methods used in situated cognition is the absence of outcome measures (Hak & Maguire, 2000). Quantitative variables used in mixed methods often focus on process over product. For example trace nodes, dribble files, and hyperlink pathways are often used to track how students interact in the environment (Shaw, Effken, Fajen, Garrett, & Morris, 1997).

The focus of this research was to study student learning approaches and their cognitive and social interactions in PBL teams that influenced their team learning cultures and their learning outcomes. Therefore, I decided to approach this investigation by using a naturalistic approach (Lincoln & Guba, 1985) that involved living with the student tribe for the entire duration of the data collection. Therefore, I decided to approach this investigation by using a naturalistic approach (Lincoln & Guba, 1985) that involved living with the student tribe for the entire duration of the data collection.

63 The methodology for this research study was chosen to suit the research questions and to enrich the notion of learning cultures that emerge in PBL teams. The concept of learning culture that emerges in a PBL team as a result of the interactions of norms, attitudes behaviours and learning approaches of its members is considered the nature of reality (ontology). As this study is set to investigate a constructivist paradigm, the overarching research strategy employed that is qualitative naturalistic inquiry will inform its epistemology - the nature of this knowledge, its foundations, scope and validity.

Most qualitative research studies that use naturalistic approaches such as participant observations to collect data use ethnography, a genre of writing, to provide descriptive accounts of people, their beliefs and behaviours from their perspective of a context (Bogden & Biklen, 1998; M. B. Miles & Huberman, 1994; Vale, 2001). This is particularly because of its remarkable sensitivity to people, culture and context to observe, interview and accumulate field notes and other materials (M. B. Miles & Huberman, 1994). Further, as ethnographic research is conducted in the natural setting, it allows selecting or sampling of groups or individuals for the study because they display characteristics that are considered to be archetypal of the group or the phenomenon under consideration (Vale, 2001).

Hence, after carefully considering many important issues such as time and effort involved, the type of data collection involved (observation, interviews, etc.) and the anonymity of the participants, it was decided that this research study would use the advantages of ethnography to study the experiences of students and their teams in PBL. As this study focuses on student experiences of PBL from their view point, it was important to observe students in their learning spaces rather than just conducting interviews. Ethnographic observation not only adds richness to the collected data, but also provides privileged insight into student attitudes and behaviours towards autonomous learning and learning in a team.

3.3.1 Participants and sampling procedures

This study was conducted at the School of Electrical Engineering at Victoria University. It is recalled here that all research procedures reported in this thesis were approved by the University Human Ethics Committee, Victoria University. Students

64 studying the PBL subjects or units of study in a first year electrical engineering degree were invited to participate in this study. This School along with the School of Architecture, Mechanical and Civil Engineering designed and implemented problem- based curricula for the first time at the time this study commenced. Further details about the course and the PBL subjects are provided in Chapter 4.

Of the 79 first-year students who were initially enrolled in the electrical engineering degree, 50 students agreed to participate in this study. Students and their PBL supervisors were observed after seeking permission from all of them to participate in observations of students in various PBL settings for example, in their PBL classroom, at the electronic engineering laboratory and at the university library. Informal interviews were conducted with the first year electrical engineering students and their teachers who provided permission for observation. A total of eight supervisors participated in this study during observation of students.

Special permission was sought from ten students selected at random for formal interviews, access to the samples of their work and for observing their participation in a simulated engineering problem-solving situation. The study was conducted over two semesters for one year and followed the same students. One of the ten randomly selected student subjects in this study discontinued the course after Semester 1. The data concerning this student was discarded. Another randomly selected student subject, who was frequently absent, indicated a desire to withdraw from formal interviews during Semester 2 citing unavailability as his reason. However, this student permitted the use of data involving his participation collected during Semester 1. This reduced the total number of randomly selected students for the non-naturalistic aspects of this study to eight in Semester 2.

3.3.2 Role of the researcher

For the entire duration of this study, I presented myself to the participants of this study as an observer. I neither taught nor assessed students in the PBL program. Some interactions with participants were expected to occur informally with students during observations in their PBL workspaces (studios). In order to avoid risks such as, students who may be concerned that I may divulge comments made by them to the PBL supervisors, all participants were informed and reminded that I was not a teacher

65 or a teacher’s spy and that I would not direct, instruct or help them do their work. They were also informed that any form of information involving them would not be shared or passed on to their supervisors unless they explicitly asked for that to happen. Participant teacher subjects, who may have been concerned that I might divulge comments made by them to the institution were also informed and reminded that the data collected in the study would not be used as a measure of their performance. They were informed and reminded that I was not a spy for the institution and any information concerning them would not be passed on to the institution.

I negotiated times suitable to students for observation and interviews. This indicates the fact that I was not present all the time. Participants were reminded that they could withdraw at anytime; leave at anytime and any data involving the individual who did not wish to participate further would be discarded if they wished.

As an observer in this PBL setting, I was exposed to student opinions about the PBL problems that they were given during their course and their concerns regarding the level of complexity, the time duration that they thought was required to solve those problems and access to the resources that they thought were required. This knowledge of student views of a problem in PBL helped me to design the problem for the simulated problem-solving activity. My role as an observer also helped me organise my ideas for analysing the data, although multiple methods were used for collecting data for this study. The data collection methods used for this study will be explained in the following subsection.

3.3.3 Data collection

Approaches to data collection in qualitative research continually expand in order to provide more instances and thereby enhance research designs (Bogden & Biklen, 1998; M. B. Miles & Huberman, 1994, 2002). According to Miles and Huberman (1994), information may be collected using four basic types for all the traditions in qualitative research including ethnography. They are observations, interviews, documents and audio-visual materials. The processes that enable these data collection methods include gaining access and establishing rapport with the participants (in this case the students as well as their PBL supervisors), recording information, exploring field issues and storing data for analysis (Creswell, 1998).

66 This research study used all four types of data collection methods: observations, interviews, document reviews and audio-visual materials. Data was collected in this study over two semesters. Data collection started at the beginning of Semester 1, 2006 and finished at the end of Semester 2, 2006.

3.3.3.1 Observation

Ethnography was the preferred method for observing the students, their attitudes, behaviours and interactions as individuals and as a team to interpret their learning approaches, learning cultures and their learning outcomes in the engineering PBL setting. The observation methods that were used in this study were trialled in another tertiary group problem-solving setting unrelated to the engineering program before observing the participants of this study. The trial provided an opportunity to strengthen the focus and the guiding questions that were used for this study.

Observations of students in this study were carried out in two contexts. Firstly, observations were carried out when the randomly selected students participated in scheduled supervised team meetings for one hour every week during both Semester 1 and Semester 2. Secondly, observations were carried out with all student participants when they met unsupervised in the PBL studios and elsewhere, for example at the electronics laboratory, the university cafeteria, the university bar and in the corridor next to the PBL studios where students conducted some of their experiments. Students in some teams had unsupervised meetings more than once a week and students in some other teams met only during supervised team meetings. At least six observations were carried out every week during both Semester 1 and Semester 2.

All the observations were recorded by taking field notes. These included descriptive and reflective notes, which are rich and thick descriptions of observations based on an observation protocol or log (Creswell, 1998). A log is a predetermined protocol that contains the research questions to focus the observations. Some examples of questions that were used as a guide to focus observations included:

• Who is present in the team?

• Who initiates the conversation in the team? • What is the team currently discussing?

67 • What activities are students involved in?

• What are the different roles that students assume? • How do they respond to their supervisor’s directions?

• Who leads the team?

• How do students communicate with their peers?

• What ideas are exchanged during such conversations? • What is the role of teachers?

A sample of the field notes taken during observation of a supervised team meeting is presented in Appendix 1. In addition, some team meetings and team oral progress presentations were video-recorded. According to Denzin and Lincoln (2003), video- recording can clearly define the sense of complexity and can assist construction of interpretations that build on the researcher’s notion and assist revisiting. It is for the same purpose that video-recording was used in this study. All video recordings were transcribed and the transcripts were used for analysis. A sample of the transcript of video-recorded observation of a supervised team meeting is presented in Appendix 2. 3.3.3.2 Interview

Alongside ethnographic observations, semi-structured individual student interviews of 20 minutes’ duration and student focus group interviews of 30 minutes’ duration were conducted to support and complement the data collected through observations. The individual interviews and focus group interviews conducted are called ‘formal interviews’ in this dissertation. Twenty eight formal interviews were conducted with randomly selected students in this study. Other participants were not formally interviewed. Focus group interviews were conducted at the beginning and individual interviews were conducted at the end of both Semester 1 and 2.

Open-ended questions that required participants to provide an elaborate answer about their experience in PBL rather than answering ‘yes’ or ‘no’ were designed for both focus group and the individual interviews. The questions for focus group interviews focussed on how students approached learning as a team in PBL and the questions for individual interviews focussed on their individual contributions and engagement in

68 team-work. Questions about their understanding of learning in PBL and their learning outcomes were also included in the individual interviews. The focus group interview questions for Semester 1 and 2 and the individual interview questions for Semesters 1 and 2 are presented in Appendix 3 and 4 respectively.

All formal interviews were conducted at a time that was convenient for the participants. All formal interviews were audio-recorded along with notes taken while interviewing participants.

In addition to the formal interviews, informal interviews were conducted with the general cohort of first year students studying in PBL. Informal interviews were conducted during observations and at other times when students socialised at the university bar. Informal interviews were also conducted at a student participant’s birthday party to which I was invited. The questions asked in informal interviews were mostly designed to clarify a student’s conversation, action or behaviour during an unsupervised team meeting or at the laboratory. Examples of informal interview questions included:

• Where did you get this circuit diagram from?

• Why are you using a Zener diode?

• What is different in this design when compared to your previous design?

Sometimes informal interview questions such as “What are you guys up to?” were asked to understand rather than assume the activities in which students were involved at the time of beginning both formal and informal observations. Creswell (1998) insisted that data collected from multiple sources and informal conversations should be written immediately to prevent loss of information. Informal interviews in this study were not audio-recorded, but were immediately written down on paper. Some informal interviews were video-recorded as questions were sometimes asked during the middle of video-recording a team meeting.

3.3.3.3 Course documents

The course documents are a source of information that direct student learning. As course documents may define the learning objectives, the expected learning outcomes

69 and learning activities including the problems and approaches that students are required to use for their course, these documents were collected for this study. Course documents collected for this study included the course brochure for the Bachelor of Electrical and Electronic Engineering degree, unit outline for the two PBL units in the first year, problem briefs, the induction manual, course timetables and PBL studio timetables. The course documents were available to all students studying the PBL subjects and were mostly collected from the University’s internet website. Problem briefs were collected at the time they were provided to students. The problem documents that were collected for this study are presented in Appendix 5 and are analysed in Chapter 4 to explore the setting in which this study was conducted.

3.3.3.4 Student work samples

The samples of student work used in this study were their portfolios. Individual portfolios were the main form of assessment for each of the PBL subjects. The student portfolio was essentially a collation of evidence for that student’s achievement of each of the prescribed learning outcomes for that subject. The portfolios of the ten randomly selected students were collected after they had been formally assessed by their PBL supervisors in the end of Semester 1. However, only eight portfolios were collected in the end of Semester 2, because two students could not be followed in Semester 2 for the reasons mentioned earlier.

It was expected that the portfolios would illustrate what students had actually learnt in the PBL subjects. Student portfolios were also used to look for evidence of student application of disciplinary and cross-disciplinary knowledge and of innovative skills. Sample data from students’ portfolios are presented in Appendix 6.

3.3.3.5 Questionnaire

Access to a report of an evaluative study conducted by the University’s Postcompulsory Education Centre using the same cohort of PBL students was requested and obtained (Gabb & Keating, 2006). This study used a questionnaire to gather data about age, previous school experience, socio-economic status, family education history, languages spoken, work status while undertaking the course, access to computer and internet from home and their main reasons for enrolling in this course. The findings of this study provided important demographic data regarding the

70 participants of this study. The findings of this study, which is presented in Chapter 4, alerted the researcher to the heterogeneity of students enrolled in the course and to factors such as their previous school and work experiences that needed consideration when analysing their learning approaches in this PBL setting.

3.3.3.6 The group problem-solving activity

This simulated engineering situation was designed to gather information about students’ application of generic skills and the disciplinary and cross-disciplinary knowledge that they had developed during the first year of their degree course through PBL. Particular attention was given to their use of communication skills, team-work