OFICINA GENERAL DE TRIBUTACION

Active music listening may be interpreted in many different ways. Volpe and Camurri (2011) distinguish between content and user centered active listening applications, whereas the first aim at providing a more creative approach to music content and the other depend strictly on user interaction or degree of users collaborativeness. To the first group belong MusicSpace (Pachet et al., 2000) where the user can manipulate the mixing of pre-recorded song audio tracks by changing the spatial position of the various instruments on a GUI interface, and Masataka Goto’s various active listening applications aimed at song chorus detection (Goto, 2003), song naviga- tion (Goto, 2007) and musical information display through a web interface (Goto et al., 2011). Interaction-based active listening relies on a single user gestures, as in the case of Orchestra Ex- plorer (Camurri et al., 2007), or on many users coordinated interactions as in the Sync-in Team (Leman et al., 2009) game, where users have to synchronize their movements to a musical beat and achieve a rewarding score according to the degree of synchronization they reach. Also in Mappe per Affetti Erranti (Camurri et al., 2008), a project developed at Casa Paganini/Infomus (Genova, Italy), the musical output is linked to the interpreters collaborative behaviour, depend- ing on the zone occupied by the dancers and on how their movements relate. Other experiments are reported in (Varni et al., 2012) where three applications based on users movements synchro- nization are described. As a matter of facts, in both application groups some action is performed on the musical content with the use of various techniques and materials. Notwithstanding, a dif-

7

See https://en.wikipedia.org/wiki/Guitar_Hero for ’Guitar Hero’ reference.

8_{The Gestalt theory provides a basic setup to understand how visual and audio perception works. It is based on}

six principles: proximity, similarity, closure, good continuation, common fate and good form, which explain how listeners organize a sequence of events in greater units which in a musical piece are organized at various levels of complexity.

62 Interactive Spaces: Models for Motion-based Music Applications

ference may be drawn between applications which employ traditional computer graphical user interfaces and applications based on gesture or full body interactions. The possibility of in- volving gestures and users movements in human-computer interaction, has opened a wide set of possibilities for interface designers (Jacob et al., 2008), one of which is the richer interaction data availability which may help to design better and more intuitive interfaces for creative listening. Moreover, acting in the reality allows the possibility of considering the collaborative element which is regarded as an important design factor in many active listening applications. Table 5.3 and 5.4 report active listening applications subdivided in GUI and gestural/bodily/collaborative or social interaction applications.

Application/Main author

Music Material Action Task

MusicSpace/Pachet multitrack audio mixing post-processing Smart MusicKiosk/

Goto

song audio track chorus selection archive navigation Cindy/ Goto song audio track dance alignment dance sequences nav-

igation Lyric Synchronizer/

Goto

song audio track/ metadata

lirycs alignment lyrics navigation INTER/Goto song audio track mixing/equalizing post-processing Musicream/ Goto song audio track searching for song

similarities

archive navigation Musicrainbow/Goto song audio track searching for artists

similarities

archive navigation Drumix/Goto song audio track adding tracks composing Songle/Goto web song audio track

platform

track analysis music understanding

Table 5.3: Table of active music listening applications based on a graphical user interface. Both tables summarize the characteristics of active listening systems with respect to the music material employed, the action performed (i.e., the result of the processes applied on the audio file) and the application’s task. Table 5.4 subdivides the interaction modality in gestural (single user), full body/social (bodily movements of a group of users) and gesture/social (coordinated gestures of two users). Though the original idea of active listening did not include any creative operation on the existing music material, “... we seek to create listening environments for existing music repertoires, rather than creating environments for composition or free musical exploration ...” (Pachet, 1999, p. 4), some active listening applications approach the idea of music creation. The Drumix application has the task of superposing a new generated drum track to the existing song tracks, whereas Mappe per Affetti Erranti, combining the emersion of instrumental parts with expressive interaction algorithms, can change very deeply the perception of the pre-recorded music material, approaching thus the manipulation levels of a real composition. The materi- als employed in active listening are mainly song audio tracks or multitrack audio. Song audio tracks undergo digital sound processes like acoustic features analysis and similarities detection (Goto, 2007) or digital filtering, whereas multitrack files are used to explore polyphonic com- positions (Camurri et al., 2007), to mix tracks in a creative way as in MusicSpace, INTER, and

Chapter 5. Responsive Floors Music Applications 63

Application/Main author

Music Material Action Task Interaction The Orchestra Ex-

plorer/Camurri

multitrack audio mixing exploration gestural Sound Scope Head-

phones/Hamanaka

multitrack audio mixing post- processing

gestural Mappe per Affetti Er-

ranti/Camurri

multitrack audio selection/ ex- pressive inter- action interpretation/ composition full body/ collabora- tive Sync-in-Team Game/Leman

regular pulse beat align- ment performance synchroniza- tion full body/ collabora- tive

Sync’n’Moog/Varni song audio track filtering recomposition gesture/ col- laborative Sync’n’Move/Varni multitrack audio adding tracks recomposition gesture/ col-

laborative Sync’n’Mood/Varni multitrack audio synchronizing recomposition gesture/ col-

laborative Good or Bad?/ Man-

danici

multitrack audio selection recomposition full body/ social

Table 5.4: Table of active music listening applications based on gesture or full body interaction.

Sound Scope Headphones (Hamanaka and Lee, 2007). A further action allowed by multitrack recording is the song recomposition, which is made by adding the different musical layers cor- responding to the various tracks one after the other till the composition’s completion. This kind of music recomposition is the task of the Sync’n’Mov application (Varni et al., 2012), whereas Sync’n’Moog employs the spectral changes produced by a digital filter to allow the whole piece to emerge. Another technique has been experimented in the Sync’n’Mood application, where the recomposition happens through the synchronization of the piece’s instrumental parts. In all these last three cases the user’s reward is the compositions’s full intelligibility obtained with users ges- ture alignment. A similar mechanism is employed in the Sync-in-Team game which is the only active listening application, except GoB itself, to be defined as a game.9 _{As an active listening}

application GoB employs multitrack audio of songs or of other musical works to recompose the original piece. The tracks are selected by one player and then accepted or discarded by a sec- ond player. The game’s aim is the music recomposition, obtained through body-operated, social interactions.10

9

According to Juul’s definition (Juul, 2011), a game is a formal system characterized by 6 main features: rules, variable and quantifiable outcome, valorization of outcomes, player effort, player attached to outcome and negotiable consequences.

10_{The word “social” instead of “collaborative” interaction is employed, meaning that here two players contribute}

to select and accept/discard the audio track in two different game phases, performing two different, separate actions, whereas “collaborative” expresses a simultaneous, aligned interaction aimed at obtaining a common result.

64 Interactive Spaces: Models for Motion-based Music Applications