In the next chapter, I present examples of works (not my own) that use projected text, thus outlining a historical practice of this particular multimedia form. Instead of organising this discussion chronologically or by genre, I use some of the ideas outlined above to structure the discussion in terms of metaphoric relations. These examples are described in terms of dialogic relations stemming from the dominant metaphorical hierarchies. Before this, I conclude this chapter by looking at a case study from my own work, highlighting the complexity of relations between three rather than two media. I will apply the analysis method of examining how the media correlate in different parts of the piece as a whole, in order to understand how attention shifts between each realignment of media at the start of the many sections. Because of the constant shifts in hierarchy, the points made in the previous sections are highly relevant.
The work in question is Subliminal: The Lucretian Picnic. Composed in 2003 for a 15- strong ASKO Ensemble, this is a 32 minute work making use of electronics and a 10x1 video projection across the front of the stage36. Subliminal: The Lucretian Picnic deals with the type and quality of information we can process at the 'subliminal' level. This exploration takes the psychological idea of 'priming'37, how an indirect exposure to a stimulus influences an emotional response, and attempts to use it as a metaphor of how meaning is negotiated in a multimedia work. The initial intention behind Subliminal was to explore the powerful connection between images, text, sound and music lying on the threshold of perception: how they affect each other's expressive potential and resultant meaning, and how our perception changes
36
Link to live recording: https://www.youtube.com/watch?v=ZJDc-gBb9rQ
37 Priming is a term used in psychology to explain how one stimulus affects the response to another
stimulus. This was shown to work most clearly within one modality in the experiments of Meyer & Schvaneveldt (1972). In the context of this subsection, I use the idea in a cross-modal sense, to explain how a projected text can subconsciously influence our understanding of something in another medium, such as music.
depending on the type and amount of information it is fed. The piece takes the form of a 'dream' essay, where fragments of film, soundtrack, music and 'psychoacoustic' electronic sounds are combined in constantly shifting polyphonic textures, to produce an overall effect of disorientation.
My fascination with what I have termed music-text-film originally sprang from an idea about subliminal messaging, perhaps even out of a misconception of it: how split-second flashes of hardly perceptible text could interfere with the audience's experience of the music. Notwithstanding, the level of subliminal effect that I had first imagined was very difficult to achieve with the standard computer-video- beamer set-up; one would need a 'tachistoscope', a projector with the capability of opening and closing at shutter speeds of about 1/100th of a second or shorter. Furthermore, there was the lingering artistic issue of whether something lying so deep under the radar of consciousness could significantly influence our perceptions. In practice, I found it more interesting to work on the conscious level of visual text perception, dealing with the subconscious level through the combination of the visual and aural.
Accounts of the first use of subliminal techniques in advertising are part of the folklore of modern media history. In 1957 James Vicary, a market researcher, claimed that over a six-week period 45,699 patrons at a movie theatre in Fort Lee, New Jersey, were shown two advertising messages: 'Eat Popcorn' and 'Drink Coca-Cola' while watching the film Picnic (directed by Joshua Logan in 1955). According to Vicary, a message was flashed for 3/1000 of a second once every five seconds. The duration of the messages was so short that they were never consciously perceived. Despite the fact that the customers were not aware of perceiving the messages, Vicary claimed that over the six-week period the sales of popcorn in the theatre rose by 57.7% and the sales of Coca-Cola by 18.1% (O'Barr 2005: 4).
These claims later turned out to be false, when Vicary himself admitted that the figures had been invented as a marketing ploy. Nevertheless, the idea of the techniques he supposedly used caused nationwide unrest at the time. People became fearful of being susceptible to this kind of subliminal manipulation, especially with regards to the new media, such as television, that was beginning to permeate the social landscape. This fed into the general climate of paranoia, in part stoked by the political reality of the time, but also because of the social and technological changes that were taking place within society (Packard 1957).
The composition Subliminal: The Lucretian Picnic is split between four basic levels of media: two visual – text and image – and two aural – live ensemble and electronic soundtrack. The visual, cinematic material consists of samples from the film Picnic. This was a commercially and critically successful romantic drama starring William
Holden and Kim Novak, about a drifter arriving at a mid-western town on Labour Day and falling for a girl destined to marry a less charming local.
The manipulation of the source film material in Subliminal is three-fold. Firstly, in order to disorientate the conventional narrative reading, disengaging the diegetic causality, the narrative order is reversed so that it does not dominate. In the second scene, for instance, we see the hero descending from a moving train and running backwards to meet a woman whose hands he grips passionately. In fact, this is the penultimate scene of the film, where we see the lovers parting and the hero running to jump on the moving train. The second form of visual manipulation is the rhythmic editing marking the temporal flow of the footage. Frame rates are manipulated and made to oscillate between positions on the timeline, creating a visual rhythmic polyphony. The third and final form of manipulation, and the one possessing the most important visual impact, is the dimension in which the film is projected. The film is cropped to a ratio of 10:1, and is subsequently projected onto a 10m x 1m screen hanging in front of the ensemble, so that what is eventually visible is predominantly a framing of hands and feet, a narrative which is restricted to bodily appendages. The idea behind this was to undermine the completeness of the original material, in order to bring it into dialogue with the music and the text, while making a reference to the visual manifestation of subtitling or surtitling.
The 'Lucretian' reference in the title is to the text used throughout the work, taken from the Rolfe Humphries' translation of De Rerum Natura by the first century Roman poet and philosopher Lucretius. His text is used as commentary on what we see and hear. It alludes to sensory experience – how the world is perceived, the nature of dreams, emotions and thoughts – with a strong emphasis on an Epicurean philosophical ideology. This becomes a fitting answer to the sense of hysteria conjured by Vicary's experiment. At times the text addresses this head-on, with notions about collective fear, the nature of dreams, reality and identity:
The type of text animation used in Subliminal varies enormously. There was a certain exploratory approach to the many rhythms and layouts used. Nevertheless, the central premise was the use of short fast-blinking text, which appears a number of times during the piece; for example in the first visual scene of the train and bus, where a text about perception appears, blinking at an on/off rate of 1:12 at the speed of 120 BPM:
In a single time, no longer than it takes to blink, our mouth to utter half a syllable, below this instant, this split second, lie times almost infinite, which reason knows as presences; and in each presence dwells its own peculiar image, all of them so tenuous that no mind is sharp enough to see them all. (Lucretius 1968: 141)
Later in the piece, two or three consecutive texts might appear at the same time, superimposed, in parallel, or scrolling quickly, with the intention to put stress or weight on the mental ability to process information in one medium against another. This was one of the principal strategies I explored in the piece: experimenting with the perceptual borders of text, image and sound, in order to investigate how much it is possible to assimilate these at any given moment when the perspective and interrelation between the media is always shifting. The viewer has to constantly reassess the perspective towards the media, changing focus in each new section. In this respect, the text narration is an extra-diegetic commentary, with no explicit connection to the visual or aural information in the work: it is there to suggest a possible interpretation of the other media. In other words, we know that the text does not directly refer to the images or the music, but it is up to the viewer to make the connections and draw the conclusions. Because it is never clear where the narrative voice is located in the narrative – who is speaking – the audience is perhaps persuaded to switch back and forth between the media in search of it.
The music itself is divided into multiple layers, which come across in differing hierarchies: 1. instrumental music, which contains quotations from the film score; 2. various electronic and instrumental pulses, which are there to emphasise or disorientate visual rhythms; 3. quasi-subliminal use of voices from the film; 4. resonances of these appearing within the ensemble or through the electronic soundtrack.
The music develops in polyphonic blocks, switching between different points of view and only occasionally finding a sense of momentum, most notably towards the final parts of the piece. This formal approach, which is intended to frustrate the audience's ability to be immersed in the narrative, is akin to a 'Verfremdungseffekt', the distancing effect associated with Berthold Brecht and found in many of the films of Jean Luc Godard, where text, image and sound are constantly obstructing one another (Monaco 2004: 136). In this case, there is no explicit voiced narration, and
what text there is functions only intermittently. We are constantly going back to the music or the visual in search of possible meaning.
Looking at the work in terms of media hierarchy, one would have to conclude that melos is the primary intended medium. The work was created for a music festival and took place in a concert hall, and there is sufficient weight on the music and soundtrack as opposed to the cropped form of the projection. However, the metaphor hierarchies change throughout the piece, specifically because 'juncture' points are used throughout to realign our perception. There are some formal structures audible in the music at times – scales, pulses, drones and loops – which reinforce the 'target-ness' of the metaphor relation, but it is not always fully autonomous. At the points where one detects the music is changing almost at the same time as the image, and that the music is directly related to the image (for instance in the dance scene at 14:18), the hierarchy shifts to the image. Both narrating 'voice' and the physically heard voice are active in shifting the hierarchy, in the form of the projected texts, but also when one hears traces of voice, sampled from the film (4:50). The 'rate of information' also works on the level of text (2:08), image (26:08)
and sound (24:38), pushing each medium to the foreground at different points. Scale of information is also a feature, specifically as the piece explores the idea of 'subliminal' levels of perception: the media sometimes lie hidden within each other. The question of media correlation in this work is complicated, because there are changing interactions between the three main media. To begin, we can draw up three radar charts, in the manner of the examples given previously, with the general overview of the three main interactions.
Music & Text Correlation:
Sync: 3
Ranges from relative sync with the music (0:00), to relative rhythmic autonomy (12:44). Never extremely synchronised, but because of the shifting block form, only really correlates with structural changes in the music.
Space: 3
There are two distinct musical spaces: live musicians and quad soundtrack. The text seems to exist in a space between these as it is projected at the front of the stage.
Scale: 3
Text size relative to music volume doesn't change that much. There is not much fluctuation in this, and in general one could say that the presence of the projected text matches the presence and scale of the music and sound.
Style: 2
As opposed to the cinematic reference in the image, the text derives from a classical source. This re- contextualisation of the text to the modern electro-acoustic sound world is not an obvious match, though, because the language contains no reference to antiquity. There is also no jarring stylistic dissonance.
Story: 4
The general narrative of the text revolves around a commentary on the senses and the nature of perception, from an Epicurean point of view. One could say that this correlates to the reductive nature of the music; the way the music is sometimes stripped down to pulses and drones. Throughout the course of the piece, changes in music to illustrate the text are consciously composed.
Sentiment: 2
In terms of sentiment, and beyond the Epicurean philosophy, there are moments when the music leans towards the emotion evoked in the text: fear, dreaminess or awe. But in general the text remains at a distance to the music, creating a critical perspective.
Text & Image Correlation:
Sync: 3
There is generally no overt sync between text and image except for the structural change, though at times the rhythm of the image oscillations correlates to the text rhythm across different sections.
Space: 4
The projection space of the text and image is shared, though the text does not originate from the cinematic material.
Scale: 3
Scale does not vary enormously between text and image.
Style: 2
The style of the text here seems to be in starker contrast to the image than the more abstract nature of the music. The type of text subtitling does not cohere so strongly with the images represented.
Story: 3
In terms of narrative, there is a stronger need to understand the images in relation to the text. Even though one can glean that the text does not explain the narrative of the film, it offers a coherent commentary on it. Some parts of the text speak directly to the images shown. For instance, at 26:55, when we read 'no man starts to act before the mind foresees its will', in the following image we see a man walking backwards. This does not explain the image, but interferes with our understanding of it.
Sentiment: 2
The sentiment presented or invoked in the images seeping through the movie narrative revolve around desire, love, jealousy and primary emotions of joy and sadness,. This is not met in the same way with text, which remains detached throughout.
Image & Music Correlation:
Sync: 4
Synchronisation in structural changes and often the timing of the oscillations in the image are in deliberate rhythmic relation to the sound.
Space: 4
The space of music and image cohere more than music and text. The music and sound make extensive use of samples and references to the original soundtrack of the film, pulling the sound into the diegetic space of the film.
Scale: 3
There is a greater variety of scale implied in the image and the sound as in relation to the text. Image can vary from close up of cropped body parts to crowd scenes. Likewise in the music, symphonic scale textures are heard next to whispered voices and close-miked single instruments. There is certainly contrast here between sections, but no overall bias.
Style: 4
Style here is more correlated than in the other relations: the soundtrack of the film is very present in the fabric of the music in the form of samples and melodic motifs played by the ensemble.
Story: 4
The narrative of the images has generally something of the same character that is shared in the music or the soundtrack. Partly because the sound is sometimes used directly from the film for that particular scene or the atmosphere of the scene matches the music in some way. The overall narrative implied by the image and sound is more complicated to perceive, because the structure is so fragmented, but within that fragmentation they correlate.
Sentiment: 4
There is more overt sentiment shared between image and music than with text, because they both contain emotive elements. There is a sense of playfulness in many scenes with the occasional shift to a cinematic relation between music and image, where music reinforces an emotional effect produced visually..
Together the three charts would line up in this way:
Music & Text: (Red) Text & Image (Green) Image & Music (Blue)
What, then, can one say about the result of these three correlation tests? We might note that the relation between image and music is much stronger here than the relation between the text and other media. This is somewhat an exception in my oeuvre, as many of my other music-text-film pieces presented in this thesis correlate much more on the level of text to music than with image.38 The strength of the music to image correlation comes from the fact that much of the music is drawn from the soundtrack of the film, either as samples or references to the orchestral soundtrack. The other reason is that the narrative voice of the text is very much 'extradiegetic', in Genette's terms: it is an outside-the-story narration, a detached commentating voice, which is not strongly reinforced in either music or image. In much of my work dealing with first person narratives, the relation of text to music is stronger, as one can read the narrational voice in the text through voices in the music, but not in this