Vástago y cabeza de la prótesis

In order to research play from a multimodal social semiotic perspective, apt research methods and methodologies are necessary. Video presents multimodal researchers with particular potentials for capturing and analysing fleeting, ephemeral dimensions of face-to-face interaction such as play, yet video also raises numerous challenges. This chapter critically considers key issues pertaining to video within a multimodal study, in particular the implications for

transcription. Literature problematizing the theoretical status of the research transcript is discussed, and the development of multimodal transcription is considered. This chapter therefore acts as a ‘bridge’ between the multimodal social semiotic theoretical perspective outlined in the previous chapter and the study’s methodology outlined in the following chapter, reflecting the particular methodological dimension of the thesis.

Video-based Research

The history of video-based social research can be traced back to the early uses of film in the late nineteenth century (Erickson, 2011), yet it is only relatively recently that video has become the “data collection tool of choice” for many social researchers (Jewitt 2012, 2). Where once the expense and size of video recording equipment prohibited its uptake and limited its fields of application, the video camera has since developed as a technology that is increasingly portable, affordable and non-specialist, offering possibilities to researchers seeking to study ‘naturally occurring’ situated interaction in contexts such as classrooms. Technological changes have also resulted in an increased presence of cameras and video in people’s everyday lives and

environments, for example through camera phones, webcams and CCTV, as well as in

professional practices. The wide and varied use of video therefore presents new possibilities not only in terms of researcher-generated data, but also video produced by research participants and the “repurposing” (Jewitt 2012, p. 3) of existing video content, for example, analysis of YouTube video (Adami, 2009) and of medical laparoscopy recordings (Bezemer et al., 2011;

Mondada, 2011).

Knoblauch et al. suggest the potentials of these new data might constitute a “video revolution”

(2006, p. 9), with Luckmann adding that new recording technologies might offer the basis for a

“qualitative leap in social research” (2006, p. 30). Despite this, it has been suggested that serious discussion of video-based approaches has “lagged behind” textual interpretive methods (Luckmann, 2006, p. 29), and that video has been somewhat methodologically overlooked (Kissmann, 2009). Whilst there is a body of work relating to visual research, particularly in anthropology and ethnography (for example Pink, 2007; Prosser, 1998), video has tended to receive scant attention in research methods handbooks, suggesting that despite increasing

uptake, methodological reflection into video-based research is “still in its infancy” (Wagner-Willi, 2006, p. 143) and is only beginning to receive particular attention (see Heath, Hindmarsh, &

Luff, 2010; Jewitt, 2012; Wiles et al., 2008).

The qualities of video make it a rich information source for capturing episodes of activity that may typically be overlooked as momentary, and fleeting, as typical of child-initiated play. Video differs from other observational and visual material, such as field notes and photographs, as it provides a temporal and sequential record, offering information about an event as it unfolds moment-by-moment, whilst preserving the sequentiality, simultaneity and synchrony of

interaction (Kendon, 1990). Furthermore, video provides a record of the fine-grained multimodal resources and practices participants draw upon in interaction, such as gaze, facial expression and movement alongside talk and its features. Video is also a “durable, malleable, shareable record” (Jewitt 2012, p. 6) with digital technology enabling repeated viewings and manipulation of temporal qualities, making it possible to revisit the video in different ways, for example slowed down, at speed, with or without sound. Repeated viewings are also made possible in different contexts, for example with participants and in research teams, enabling new listenings, new viewings and new interpretations (Lapadat & Lindsay, 1999).

Despite the potentials of video, its constraints and challenges must also be considered. The detail of video means that “a few minutes of recording produce a large quantity of visual, kinaesthetic and acoustic data” which might quickly become unmanageable and bewildering (Knoblauch et al. 2006, p. 14). Crucial decisions are often made in the moment, such as when to switch the camera on and off, what to focus on, whether to track children or remain in a fixed position, and whether to film at close range or from a distance, resulting in inevitable gains and losses in the video representation of the event. Choosing to observe at close range, for instance, might capture the fine detail of facial expression and gesture but inevitably loses the wider action. Recording decisions are often made in situ and in response to the action as it unfolds, highlighting video as a necessarily selective perspective on events (Jewitt, 2012), mediated by the technology of the camera (Duranti, 2006).

A criticism often levied against video research is that it distorts the situation and behaviours being scrutinised and so diminishes its empirical value. An opposing although equally problematic assumption is that video is entirely unobtrusive and provides a straightforward replica of reality. Both perspectives highlight a somewhat problematic perspective on reality and objectivity which has increasingly been challenged in favour of positioning video as a reflexive tool in the research process (Heath et al., 2010; Jewitt, 2012). For instance, in their work investigating midwifery home visits, Lomax and Casey (1998) studied the ways in which participants oriented towards and away from the video camera, seeing these moments as offering insights into informal/formal and shared/private boundaries in a particular context. From

this perspective, the camera’s role in the social situation is not seen as creating ‘good data’ or

‘bad data’ but as a point of investigation.

A particular challenge raised by video-based research, and multimodal research in particular, is the issue of transcription, which has been identified as an area in need of further methodological discussion (Bezemer & Mavers, 2011; Davidson, 2009; Flewitt, 2006; Flewitt, Hampel, Hauck, &

Lancaster, 2014; Knoblauch, Schnettler, & Raab, 2006). Transcription in social research usually refers to the process of turning a strip of spoken language into writing for purposes of analysis and dissemination of findings. However, if the research interest is not just spoken language but multimodal interaction, the issue of what and how to transcribe becomes particularly complex and necessitates further exploration. Before considering multimodal transcription in detail, it is valuable to first consider the status of the transcript in research.

Theoretical Perspectives on Transcription

The etymology of the word ‘transcription’ highlights its history and commonly attributed status in social research, translated from Latin as ‘thing copied’ and derived from the verb ‘scribere’, ‘to write’. Lapadat relates different approaches to transcription to epistemological assumptions and research paradigms, suggesting that positivist traditions tend to treat transcription as a relatively

‘straightforward’ intermediary stage of research, whereby speech is turned into writing as preparation for analysis (Lapadat, 2000). This is distinguished from research adopting a constructivist stance, where attention has increasingly been drawn to transcription as

representation (Lapadat & Lindsay, 1999). Rather than taking transcripts as transparent, it has been highlighted that the process of transcription will inevitably leave the transcriber’s

interpretive, analytical and theoretical “fingerprints” on the resulting transcript (Tilley, 2003, p.

752), and that as such they are academic artefacts (Bezemer & Mavers, 2011), shaped by theory (Ochs, 1979), political perspectives (Bucholtz, 2000) and professional vision (Goodwin, 1994).

Researchers have drawn attention to the process of transcription as one of translation (ten Have, 2007) and transformation (Duranti, 2006). Those adopting a multimodal approach have described this process as transduction (Bezemer & Mavers, 2011; Cowan & Kress, 2017), where meaning is remade across modes (Kress, 2010). Alternatively termed ‘transmodal redesign’ (Mavers, 2011), particular attention is given to the change in form and meaning when shifts across modes occur (for instance, when speech is represented as writing, or writing is remade as drawing). Bezemer and Mavers suggest that transcripts are therefore “transducted and edited representations through which analytical insights can be gained and certain details are lost” (2011, p. 196), reiterating Ochs’ suggestion that selectivity in transcription both results from, and in turn generates, theory (1979).

In light of these inevitable gains and losses, questions might arise as to whether to transcribe at all. Particularly in light of the increasingly digitised dissemination of research, such as digital theses, webtexts and online journal content, it is increasingly possible to include extracts of the original video recording as part of the text or in digital appendices. However, Erickson’s

perspective on video urges caution, suggesting that video itself is not data, but an information source from which data can be identified (2011). As such, transcription might still remain an important “noticing device” (ten Have 2007, p. 95) or “discovery procedure” (Duranti, 2006, p.

307) in the social research process, and a means of making the researcher’s data and analysis shareable, and so open to scrutiny and discussion.

In relation to new technologies, the primary challenge may be whether transcription systems designed for ‘old technologies’ such as the audio recorder can effectively represent material recorded by new technologies such as the video camera (Norris, 2002). Slembrouck considers the digital formats transcripts can now take, suggesting that online publishing of digital data means that the recording and analysis can remain open to inspection from both researcher and audience (2007). Silver and Patashnik compare options for computer assisted qualitative data analysis (CAQDAS) software when working with audio-visual data, including discussion of transcription (2011). They suggest that current CAQDAS packages are lacking particularly in relation to non-code-based approaches and representing data in non-linear formats, requiring researchers to reflect carefully upon their research needs when considering using the available software tools (Silver & Patashnik, 2011, paragraph 83).

Coates and Thornborrow note that in light of different researcher perspectives and decisions about how to transcribe, the same piece of video or audio material might be transcribed in many different ways (1999). Variation in transcription, choosing between what is illuminated and what is obscured, the authors consider “not a tragedy but a necessity” (Coates & Thornborrow, 199, p. 596). As Duranti notes, it is neither possible nor necessarily desirable to represent an original instance of social interaction in its entirety (2006). Ochs notes that transcripts containing too much information and detail are difficult to follow and assess, suggesting “a more useful transcript is a more selective one” (1979, p. 44). The process of selection might therefore be positioned as part of analysis, selectively engaging with video in ways shaped by the researcher’s interest and theoretical position.

In addition to analytical insights, transcripts have a rhetorical purpose, acting as ways of developing an argument and leading the reader through the research findings. In this way, transcriptional choices must be made about not only what is included in the transcript for purposes of analysis, but also how it is represented in order to be accessible to a particular audience. Considering the enduring print-based tradition of academic journals, this usually calls for transcripts that can be presented on the printed page or page-like screen, where form might

also be shaped by publishers, editors and the technological constraints of the medium (Slembrouk, 2007).

The growing recognition of transcription as a transformative process shaped by theory and researcher purposes has led to calls for increased reflexivity, examining transcription as a key aspect of social research (Bird, 2005; Davidson, 2009; Roberts, 1997; Vigouroux, 2007). Whilst there is a growing body of literature related to the practice of transcription, including theoretical perspectives (Bucholtz, 2000; Judith Green, Franquiz, & Dixon, 1997; Ochs, 1979; Roberts, 1997) and explication of conventions (Du Bois, 1991; Jefferson, 2004; McWhinney, 2000), it has been noted that there is often an absence of discussion about transcription in research project reports (Davidson, 2009; Stelma, 2009). Lapadat and Lindsay suggest that too often the focus has been on seeking to standardise transcription techniques at the expense of methodological researcher reflection on the practice of transcription itself (1999). My thesis seeks to address this issue by critically examining the process of transcription and embedding reflexive discussion of transcription within each of the case study chapters that follow.

The Challenge of Multimodal Transcription

As this review highlights, transcription has increasingly been problematised and scrutinised, yet the complexities of transcribing are multiplied further when the researcher is interested in multimodality and seeks to represent data in multimodal forms. As Bezemer and Jewitt note:

Where first we only had to attend to the gains and losses involved in a move from speech to writing, now we also need to address gains and losses resulting from a move from gesture, gaze, posture and other embodied modes of communication to image, writing, layout, colour and other graphic modes available in print. (2010, p. 187)

Whilst some disciplines have developed highly specialised forms of transcription (e.g.

Conversation Analysis) and while practical fields make use of their own notation systems (e.g.

Laban dance notation), multimodal transcription has only recently become a subject of discussion, and there is certainly no canonical tradition. Slembrouk suggests that rather than adopting established transcription systems based on their widespread usage, researchers interested in analysing interaction ought to “engage afresh with the question of notational conventions and representational-interpretive priorities” (2007, p. 823).

Researchers adopting multimodal perspectives have attempted multimodal transcription in a multitude of inventive ways. Designs have included combinations of devices such as video stills (Baldry & Thibault, 2005; Gillen & Cameron, 2017; Heath et al., 2010; Norris, 2004), drawings (Bezemer, 2008; Goodwin, 2000; Mavers, 2009) and use of musical notation (Erickson, 1996;

Falthin, 2015; Ideland, 2013), variously making use of different layouts such as timelines, vignettes, scores and grids. Transcripts themselves have also been the focus of academic

discussion and analysis. Flewitt, Hampel et al. (2014) discuss a selection of approaches to transcription of dynamic data from research fields including applied linguistics, visual

ethnography and computer-mediated communication. Bezemer and Mavers (2011) use a social semiotic framework to account for transcripts as artefacts, comparing transcripts dealing with video from disciplines including conversation analysis, discourse analysis, social semiotics and micro-ethnography. Authors have also undertaken to describe and share their own approaches to the multimodal transcription of video (Baldry & Thibault, 2005; Bezemer, 2012; Norris, 2004), and have discussed the manipulation and fixing of transcripts into different forms at different stages in the research process, showing the variation possible in transcription within one study (Gillen & Cameron, 2017; Mondada, 2007). Such studies suggest that variation is a

characteristic of multimodal researchers seeking to transcribe video, necessitating a need to reflect carefully upon the gains and losses of each transcript’s particular design.

Ochs proposes that a greater awareness of transcription means not only will we be able “to read much more off our own transcripts, we will be better equipped to read the transcripts of others”, in turn helping our evaluation of interpretations of data (1979, p. 72). In understanding new technologies and the process of transcription, Duranti suggests that “we need to find out and then clarify what one type of inscription can or cannot capture and what another type of

transcription can or cannot reveal”, by critically reviewing our own and others’ transcripts (2006, p. 306). This thesis seeks to turn attention towards a particularly challenging aspect of

multimodal research, sharing my own attempts at transcribing the multimodal activity of child-initiated play through situated examples of transcripts of a range of play episodes, accompanied by reflexive commentary.