The first task was to identify suitable sites within the College of Law for the case studies. Given the nature of the research questions, it was determined that two differing sites would be desirable: a single site may prove overly narrow and reductive, and more than two may generate excessive data or, given the immersive nature of the work, may limit possible research engagement with the sites. It was also resolved that a period of three semesters would be optimum to conduct the study, providing sufficient time for the study to develop with three successive action research cycles. In essence, the two sites to be researched needed also to represent a purposeful concept sample, that is potentially information rich and that allowed a clear understanding of the phenomenon under investigation (Patton, in Cresswell 2005). Based on this broad framing, the following specific criteria were developed by the researcher for discussion with a range of program convenors to determine site suitability:
a) a coherent teaching program with a range of subjects with differing student cohorts b) a relatively stable teaching team with experience in conducting, and responding to,
student opinion surveys
c) a willingness for the academic teaching team to actively engage in a CHAT-based, action research project over at least three semesters
d) demonstrable focus on innovative or disruptive pedagogies which may or may not have impacted on student feedback outcomes
e) capacity to further develop curriculum, teaching strategies, course materials, learning technologies and assessment based on the outcome of research
f) openness to further develop the individual and collective pedagogical capabilities of academics based on the outcomes of research
g) agreement for the outcomes of the research be investigated and published (subject to appropriate ethical clearances and informed individual participant consent)
Based on these criteria, two suitable programs were identified and subsequently offered by program convenors as case study sites for the research. The site of the first case study was the recently formed Migration Law Program. This Program is primarily focussed on delivering the Graduate Certificate in Australian Migration Law and Practice,a significant course with approximately 500 student enrolments per year. The second case stud was the ANU Legal Workshop, which offers a specialist program for
law graduates centred on professional legal education for practice. The Workshop’s core program –the Graduate Diploma in Legal Practice(GDLP) – is a mandatory
qualification for access to a legal practice certificate. Although it had been offered for over thirty years, its mode of delivery had been recently radically reformed to a blended learning mode using a combination of face-to-face and online teaching. It had around 1100 to 1400 student enrolments in recent years.Several further potential sites for case studies were identified but were discarded, as they either:
could not effectively support the collaborative action research model being proposed
had specific situational limitations that would prevent investigation of the current use or prospective use of student opinion
were constrained in their capacity to develop programs or the capabilities of academic staff based on research outcomes, for a range of differing reasons
Put simply, given the nature of the research proposed, the two case studies were selected as they afforded the best opportunity to understand the use and potential of student feedback, whilst at the same time possessing a genuine interest in developmental improvements in program design, teaching and academic capabilities. One important aspect of the recently revised ANU policy framework also assisted in facilitating this approach. This was the continuing ability of individual programs within the ANU to develop specific local strategies to seek student feedback outside the conventional quantitative mode. This afforded this action research approach in the two selected sites, and allowed the broad exploration of different approaches to the collection and use of student feedback data.
Although these two College of Law programs sat within the same broad discipline and in single College of the University, they embodied the policy and procedural approaches of the broader university (and the sector more generally) in regard to the use of student feedback-based evaluation. Both programs had previously employed the standardised ANUSET student opinion surveys, and were preparing to move to the new ANU student feedback system (as discussed earlier in this chapter). Therefore, the relevance of these programs lay not in their specific discipline or location, but their employment of broadly standardised quantitative student opinion surveys and the related mandatory
responsibility to respond to its outcomes. In addition, both programs were actively seeking to:
improve teaching and assessment quality using a collaborative action research framework
wished to identify and act on opportunities for substantial program and academic development
were open to forms of development they may be generated by collective assessment of mediated student opinion
As Norton (2009) argues, action research in university settings is most effective when it is a result of a perceived need for enquiry into what is already being done, rather than imposed as a formalised staff development initiative. For this reason, the action research was clearly framed around the history and trajectory of the individual programs rather than as a generic research initiative being bought to bear on the program for purely academic interest.
The two case studies foregrounded in this study also represent instances of the rapidly changing environment of higher education, sharing the characteristics of:
large-scale teaching programs with complex curriculum and rigorous assessment demands in a broad discipline domain
offering teaching and assessment in mixed modes of delivery (i.e. both face-to- face and online)
being under considerable pressure to recruit and retain students, maintain high levels of student satisfaction and meet rigorous expected graduate capabilities in the emerging Australian higher education ‘marketplace’
operating under various demands of institutional accountability, program and academic responsiveness and broader pedagogical effectiveness, with all of which student feedback influences in one form or another
However, they also have key differences that are important as they create a distinctive character for each case:
one program is primarily offered via online learning with limited face-to-face orientating seminars, whilst the second carries a more significant face-to-face component (though with considerable with online elements)
one program has highly diverse student demographics and academic entry levels, whilst the second has a more homogenous cohort with a standard academic entrance expectation
one has a large casual teaching group (most of whom also work in professional practice) and a small core full time academic staff, whilst the second has
primarily a permanent teaching workforce of full-time academics, supplemented by a cohort of casual teachers from a variety of backgrounds
one program had developed and modified curriculum from scratch over the last five years (within a mandated competency framework), whilst the second has an accumulated history over two decades with relatively stable curricula (and has shaped the broader curriculum framework used across the sector)
In CHAT terms, these shared and distinctive characteristics of the two programs provided the opportunity for the contextual exploration of activity settings that are discrete but are also what Yamagata-Lynch (2010) describes as ‘highly interrelated bounded systems’ (p. 79). This provides the ability to conduct sociocultural analysis of the outcomes of the cases that is multi-dimensional, and allowing a greater
understanding of the effect of individual and shared agency in activity.