In our literature review, we have seen numerous works that computationally analyze the interaction between the conductor and the orchestra. In all cases, however, this analysis is done in controlled environments and focusing on some particular aspect, such as the synchronization of the orchestra with the movements of the conductor. In our case, since the analysis is motivated by the activity that users will replicate in conducting systems, we wanted to take this analysis to a real performance. This, of course, implies some problems, since there are no variables under control and we must limit ourselves to observing possible relations between conductor’s movements and music.
For this reason, we wanted to approach the analysis knowing those specific aspects on which we could expect some correlation between the conductor’s movement and the resulting music. From an interview with professional conductors and students, we concluded that the possible aspects to be analyzed were the communication of tempo, dynamics and articulation; being aware that the conductor is not necessarily constantly conveying information of these parameters.
Accordingly, we performed an analysis of the afore-mentioned recording focusing on repetitions of the same motif (the Ode to Joy from the 4th movement in Beethoven’s
9th symphony) that appears with different variations throughout the piece. The main
conclusions of the analysis, presented in Chapter3, were the following:
• The descriptors that best capture the synchronization between the movements of the conductor’s right hand and the music beat are those obtained from vertical movement.
• The lag observed between different descriptors computed from hand movement and musical beat varies across different fragments. This suggests that automatic beat estimation from the conductor’s movement must incorporate contextual in- formation.
• As suggested in the interview, it is not always possible to find correlations between conductor’s movements and music.
• In terms of loudness, when this correlation exists, the quantity of motion seems to be the feature that best describes it.
The scores of the analyzed excerpt are included in the online repository. The complete results for beat analysis of each excerpt can be found in AnnexB.
7.4 Contribution 4: Further understanding of conductor-orchestra interaction
7.4.1 Limitations and future work
The first limitation of our analysis to keep in mind is that it is focused on a specific performance, with one conductor and one orchestra. In this sense, works that seek to approximate a general model of the director-orchestra interaction, should do so from data that represent greater variability. Focusing on specific aspects of our analysis, there are also some limitations that must be mentioned.
First, although it had appeared in the interview as one of the possible aspects to analyze, we have not dealt with articulation. The reason is that it is difficult to find similar fragments in the symphony where articulation varies. In this respect, one aspect to be taken into account when planning new recordings for this type of analysis is to, as far as possible, select the repertoire according to what is going to be observed.
Second, regarding the observation that the lag between different descriptors computed from hand movement and musical beat varies across different fragments, we have not explored the possible mechanisms that may help to predict this effect. This is, in fact, an area open to future research: is there any way to determine, from the context, how beat must be predicted from motion? The context in this sense can be the musical one, i.e. the instruments that are playing at that moment, the current tempo, the possible changes of tempo that are going to occur, etc. But the context can also refer to how the conductor’s body is moving, i.e. the speed at which the arms are moving, the amplitude of the gesture, etc. We believe that a more detailed analysis that takes into account these factors can help to better understand the observed phenomenon.
Third, in our analysis relative to loudness we have used audio descriptors as ground truth. The loudness is however a complex perceptual phenomenon, particularly in classical music. For example, in our analysis we have observed parts where the height at which the director placed his hands was correlated with loudness. However, it is possible that this effect is due to the fact that, in those fragments, the conductor was giving indications to the choir, placed in the back part of the orchestra. In this context, we believe that potentially interesting analyses can be made using other information than the loudness extracted from audio. For example, having aligned score, it is possible to know how many musical sections are playing at any given moment. Also, scores usually include dynamic annotations on how they should be played (e.g. fortissimo, pianissimo). The shared dataset is a good resource for such analyses.
In any case, a general caution that must be taken when analyzing a performance is that it is very difficult to determine whether the observed correlations involve causality. We are not here discussing that the conductor actually controls the orchestra during
the performance. What we are referring to is that, according to our observations and suggested by the conductors in the interview, actions performed by conductors during performance are influenced by a high number of factors difficult to determine. Something that we have observed during this project, although it has not been part of the presented computational analysis, is that the aspects that conductors emphasize during pre-concert rehearsals are also emphasized during the concert. For example, when a conductor asks to repeatedly rehearse a part where there is something he does not like, it is more likely that in the concert he will emphasize the requested correction with his gesture. Also, regarding the loudness analysis, one of the indications we got in the interview with conductors was that sometimes a conductor may make an indication (e.g. play forte) at a point, not needing to give that indication again for the time that indication applies (and as long as the orchestra plays according to the conductor intention). We believe in this sense that both ideas suggest that a potentially useful approach for this kind of analysis might focus on these key points where concrete significant actions occur. In accordance with these ideas, and also based on the outcomes and conclusions from other parts of the thesis, we believe that in terms of DMI mapping design it is better to focus on the final user than trying to transfer knowledge from the analysis of conducting performances. It is highly complicated to find causal relationships between conductor movements and music in an uncontrolled environment. Also, what users of a conducting system will do will always be very different from what a conductor does at the concert. Finally, we would like to point out that conductor-orchestra communication is not con- fined solely to body gestures. Gaze and facial gestures also play an important role in expressive communication. Future works that deal with this aspect might refer toPoggi (2002), who describes a “lexicon of the conductor’s face” including gaze, head movement and facial expression gestures.