Two months after the final workshop, the researcher invited all of the original 26 teachers who had participated in the action research to participate in a semi-structured interview. Of these, 13 teachers accepted the invitation. Perhaps unsurprisingly, those who took up the offer had been involved in the three semesters of the action research. Six of the interviews were conducted face-to-face and the remaining via telephone. In order to understand the context of participant reflections, the interviews sought to initially explore the levels of teaching experience of the participant, some of the influences that had shaped their current approaches to teaching and the affordances and hindrances they perceived to initiating pedagogical change. From here, the primary focus was moved to experiences with student feedback and specifically their reflections on the action research model.
Broadly, the teachers interviewed roughly divided into two categories: teachers who had been teaching since the inception of the program in 2006 (with around six years
experience in the program) and the remainder with two to three years experience. Within these two cohorts, there were differing professional backgrounds: around half were migration law practitioners who melded this with their academic work, whilst the remainder were experienced in the field but currently only worked as full or part time teachers (as well as researchers and/or public policy advocates) in migration law.
These differing origins were clearly reflected in how participants responded to most questions. An example of this was to the question of what had primarily shaped their approach to teaching migration law. Those who maintained a migration law practice were strongly shaped by the need to develop an appropriate and robust array of skills and interpersonal capability for the profession. Alternatively, those outside immediate practice tended to focus on the need to effectively educate, inspire and/or challenge students around the role and purpose of migration law (and therefore migration agents). It was also apparent that those who had been involved with the program from the beginning carried a somewhat more sophisticated understanding of the challenges of teaching curricula which included a difficult and regulated combination of legal, practical and interpersonal knowledge in a blended delivery mode. Similarly (and unsurprisingly) those teachers newer to the role generally reported the most change to their teaching approaches over time. However this was manifested more in regard to functional use of the online mode of delivery and in preparation students for
assessment, rather than in broader pedagogical domains.
In further background to the specific issue of student feedback, teachers were asked to reflect on what they perceived to be the most significant constraints to improving the effectiveness of their teaching. Interestingly, practitioner-teachers universally identified a lack of teacher education as the most significant. Conversely, those more experienced in teaching largely cited a lack of available time and resources as constraining. Both categories of teachers were however anxious about the unbounded potentiality of emerging learning technologies (including some of those currently in use in the
program). Several developed this further to express that these technology challenges - in combination with perceived onerous assessment demands -were making pedagogical innovation a fraught proposition. The dual pressures of limited time, technical skill and high regulatory scrutiny of student learning outcomes created significant apprehension
and acted as a powerful constraint to innovation. One teacher (a practitioner-teacher) succinctly captured this range of anxieties and their constraining effects, commenting:
Time limitations are a big issue - that is, the limited time for preparation for each course, in combination with the speed at which we need to move through the material in an intensive form makes changing teaching difficult. Although getting training and confidence in all the technologies available might help develop my teaching, I don’t really know if I could find the time to develop and use it
effectively. And the (regulator) insists on an exam at the end and this really limits what we can do…we know being an agent is much more than this, but if the students do not pass the exam we are seen as being poor teachers. (ML-8-4)
Participants were also asked to reflect on their previous experiences with quantitative forms of student feedback (the ANUSET system). For the inaugural teachers in the program, this experience was over six semesters (and for some longer where they had taught in other areas), and for newer teachers only over two or three semesters. Yet all but one of the thirteen participants reported negative or null experiences with
quantitative student evaluation feedback. Several respondents remained sceptical:
…a lot of surveys and not much use….it seems they were for bureacratic reassurance rather than to improve our teaching. (ML-8-4)
…you got the impression that that as long as not too many students are complaining and everything is done on time, then ANU is happy. (ML-8-7)
…the sole focus seemed to be recording student feedback as the only way to ‘really’ evaluate (the effectiveness of ) a program…this seemed more a process than an action. (ML-8-13)
Other respondents doubted their real value in providing insights into teaching quality:
...these (quantitative) evaluations, because they were really not aggregated or analysed, have not been particularly useful in guiding us as teachers….to know how to improve the program and our teaching. (ML-8-2)
…individual comments from students give some clues, but it hasn’t really been possible to know whether or not that comment is representative of many students’ experience. (ML-8-5)
…as far as I knew, students completed their questionnaire and that was about it….no real impact unless a real problem was apparent. (ML-8-1)
One respondent adopted a different posture on qualitative student feedback, best captured when they observed:
…we need to hear clearly the student perspective no doubt, and understanding this in the context of what’s happening in other courses and comparing results. This means teachers are forced to think hard about what they are doing, particularly if its poorly regarded by students, and whether they should do things differently. (ML-8-3)
Nevertheless, what was apparent from all respondents was a genuine recognition of the important role of student feedback could perform in improving teaching and enhancing student learning. For most this was grounded in a commitment to create a productive learning environment, as well as produce graduates capable of contributing positively to migration advice and advocacy (though with varying emphases and characterisations). Some representative observations on this were:
…my objective is to assist students understand some fundamentals of migration law and practice. So I want students to engage in the course and appreciate what they have learnt during it. (ML-8-6)
…I really would welcome more opportunities to evaluate the program – in
whatever way it needs to be done – and make changes that improve their outcomes. (ML-8-10)
….as I spend a lot of time guiding students through the subject, answering questions, highlighting the relevance of critical components and clarifying the areas students are having difficulty understanding…it is critical I know how appropriate the judgments I make on these matters actually are from a student perspective. (ML-8-3)
Respondents were then asked to reflect on the action research model they had
encountered over the three previous semesters. All respondents were broadly positive about the action research model, albeit with varying levels of enthusiasm. Six of the respondents offered a highly favourable assessment of the model. It is notable all those in this category were primarily the teachers who were part of the original group of
teachers recruited to the program. They included both full-time and part-time teachers. Some observations that characterised this group included:
…..very useful and I have taken a lot on board and changed (my teaching) to reflect that. (ML-8-6)
…it was extremely useful, part of that was seeing the evaluation of the program overall and not only the individual courses. (ML-8-11)
…of particular value was how the (action research) based evaluation identified key questions based on student responses, as well as some potential responses. These were a brilliant springboard for the review sessions. (ML-8-9)
…it was the first time that an attempt was made to provide feedback in an organised manner. It was useful in that it challenged me to consider some of the harder educational issues involved, when I hadn’t really been previously aware of them. (ML-8-8)
Other respondents were less certain. Although they saw the potential value of the action research model, they were somewhat more equivocal about how realistic it really was given the time limitations between review sessions and the recommencement of teaching. Some of these sentiments are represented by these observations:
….it was great to sit down and spend some time and discuss what worked and what didn’t. However we needed more time to actually think through and implement what was decided was necessary or useful. (ML-8-3)
…I would have liked more time to go back over the recommendations and evaluation report to see what more I might do. At times it seemed we had so much data that it was very challenging to prioritise it, let alone act on all of it. (ML-8-1)
…quite useful, but for me it reinforced many of my perceptions, perceptions that have been difficult to really act on given my limited time and resources etc. (ML-8- 12)
Other respondents, though positive about the model, saw it as part of a useful enterprise that was more general and not necessarily unique to this form of enquiry:
…quite useful, though I’m not sure it didn’t tell us anything we didn’t really know if we had considered it at this level of detail. (ML-8-2)
…reflection is always useful, though this work did use considerable resources and really needed someone to be co-ordinating and driving it if it was to succeed. (ML- 8-7)
…I got some useful information, for instance understanding the online lurkers and problems with our assessment and feedback. These are things that are useful to know. (ML-8-10)
However, all respondents agreed that the focus on evaluating student learning (as opposed to more conventional focus on evaluating teaching) was a useful enhancement. It was also universally regarded as providing a more legitimate basis for determining the quality of teaching pedagogies than conventional quantitative ratings. Yet respondents were more equivocal on how influential in actual practice the action research outcomes were. Around half the respondents provided substantial examples of how the action research model outcomes had impacted directly on their own teaching approaches. This is captured well in these observations:
…the (action research) was quite influential. I have implemented changes. For example, better setting up of student expectations and trying to scaffold and support student assessments. I now highlight the relevance of certain activities and relate learning more to migration agent practice. (ML-8-11)
…I was inspired to think more clearly about my expectations and those of the students, how to integrate the worlds of learning and practice and how to ensure students were learning for assessment, and what is the most feedback, like rubrics. (ML-8-8)
…it did change the way I looked at myself as a teacher and forced my to reconsider habits I had developed. (ML-8-6)
Other respondents were less convinced about the actually impact of the action research on their teaching approaches. Notably most of these responses came from those who were part-time teachers simultaneously engaged in professional practice.
…it was only moderately useful. It certainly raised issues but the question was how much could realistically be achieved in the time available. (ML-8-1)
… reasonably influential I guess, but having said this I found it actually
reinforced the way I was headed anyway, so it didn’t provide a direct impetus for change, but a motivation to continue. (ML-8-3)
….it has been useful to better understand the process and the impact of teaching and supporting students. But I also think it put a lot of pressure and maybe unrealistic expectations on those of us who weren’t here all the time to do more in our own time. (ML-8-7)
Conclusion
The CHAT-based, action research model used over three semesters in the Migration Law Program was moderately effective in sustaining engaged professional dialogue and in generating some tangible developmental change. The model over its life generated three substantial Evaluation and Course Development Reports, around 60 000 words of qualitative student feedback, some significant evidence of pedagogical improvement and a modest range of situated academic development. Equally, it was not fully effective. The original model proposed engaging teaching academic reflections alongside that of students reflecting on their learning. This proved largely ethereal during the three semesters, with reflective dialogue by teaching academics was largely confined to the pre and post semester workshops.
Further, as was reported in participant reflections at the end of three semesters, there was some uncertainty about how influential the model was in practice. Its initial introduction proved challenging due to the complex conceptual framing of the model. Its broad collective nature also created some early reservations with the lack of specific focus on individual subjects and teachers. The nature of this teaching workforce, involving a small core of conventional academic teachers and a second larger group of practitioners from the field engaged in teaching, had divergent responses to elevated student feedback which seemed to limit its potential developmental impetus. Moreover, the significant time and resource limitations of the primarily part time teaching
workforce, in tandem with an unexpected fall in enrolments and staff in the third semester, made it difficult for the action research to gain genuine momentum.
The next chapter reports on the second case study. Following this chapter, a more comprehensive analysis of this case - in the broader context of the two case studies - is offered.