During the first phase, I conducted two observation periods of Momentum’s assessment panels and an observation of the shortlisting process for applications in one round. The Momentum assessment panels occurred after specific funding deadlines, of which there are typically four per year, and consisted of panel meetings where the applications were discussed and funding decisions made.
Access was granted in 2015 to observe panels for two funding deadlines. Due to the need to maintain anonymity of the advisors in attendance and the applications discussed, I will not specify the exact rounds or dates. This is intentional, since it enabled me to make the best use of time and resources, and to allow the
collection and initial analysis of different datasets to inform the further collection process. The research aims to maintain consistent reflexivity. Being a reflexive researcher requires the researcher to be tuned into her own values,
predispositions, context and biases, and how these can impact the research (Bryman, 2008: 682). In practice, I have reflected and taken an iterative approach during the data collection, analysis and writing up processes to understand my situated knowledge and its implications for my research. The overlap of actors and collection methods sought to facilitate this. Also incorporating a reflexive approach in observations, I will now describe the simple or ‘unobtrusive’ observation method employed within Momentum assessment panels, whilst also incorporating a
reflexive approach in observations.
UNOBTRUSIVE OBSERVATIONS: SIMPLE OBSERVATION METHOD
Unobtrusive, or non-participant observation, was suitable within the case study design, and I determined that unobtrusive observations would be the most suitable option for gathering data on Momentum decision panels. Unobtrusive research aims to limit reactivity, which is “the extent to which the presence of an observer affects the behaviour of those observed” (Lee, 2000: 47). While potentially unavoidable, having a high degree of reactivity on my part as an observer in the Momentum decision panel would have severely limited the reliability of the data collected. One reason for using unobtrusive observation methods, instead of
74
interviews or questionnaires, is “because respondents commonly try to manage impressions of themselves in order to maintain their standing in the eyes of an interviewer,” and it is thought that impression management may be harder to maintain in non-participant observations (Lee, 2000: 34). This is twofold, as Lee explains:
Some writers have suggested that the situational demands facing participants in some contexts are sufficiently engrossing to override a subject’s awareness that an observer is present. On the other hand, maintaining a pattern of activity designed to present to the observer a sanitized view of what goes on in a setting requires considerable effort on part of those observed. In many cases the effort becomes too great after a time or those observed simply forget about the presence of the researcher.
(Lee, 2000: 47)
In the Momentum panels, the demands placed on advisors and PRSF staff are great. Staff must play sound and video clips submitted with applications, take notes on the advisors’ comments in order to feedback to applicants and chair the panel, keep it running on time, and help advisors to reach clear decisions. Panels lasted roughly four hours, with seven or eight artists selected from a pool of 75-80.
Typically, there is little room to consider what an observer is writing8, which I have experienced when I chaired and ran assessment panels for other funds. The advisors, on the other hand, are trying to remember what they liked or did not like about applications and negotiating their opinions with those of others.
I chose to use unobtrusive methods because I wanted to discover the ways in which decisions were made and how the dynamics of the room and interplay between advisors related to the conversation topics and funding decisions.
Interviews or other self-reporting means would not provide the best means of capturing interactions between individual advisors (Lee, 2000: 34). Neither questionnaires nor interviews could provide me with insight into the dynamics of the room. Sitting in the panels allowed me to see how a decision not to fund an application might be made initially in a panel only to be changed later on, spurred by the impassioned plea of one advisor. While that was rare, the dynamics of the room were important. If interviews had instead been pursued after the panels,
8 I was permitted to write notes during the Momentum assessment panels but was not allowed to record audio.
75
there would be a risk of advisors misremembering these occurrences, like stating they had always supported the funding of a particular application when, in the panel, they had originally been reticent. These occurrences all have implications for questions of value and the ways that value is expressed and defined within decision panels. From an access perspective, unobtrusive observations of decision panels provided the easiest access to participants and allowed the participation of all of the members of the panel, whereas pursuing interviews or sending out a questionnaire could have led to limited responses and a less holistic picture of the decision-making process (Lee, 2000: 4). The aim of the observations was also to limit advisors’ reactivity to me, the researcher, and unobtrusive
methods provide a suitable means of achieving this.
My approach to unobtrusive research did not include researching
participants in public or covertly, as some researchers have done. Care was taken, however, to limit reactivity and the impact of my presence on the Momentum
assessment panels being observed. During panels, I sought to make myself one of the group but in a passive way and without being too uninvolved as to appear conspicuous (Lee, 2000: 46). I sat with the group but did not speak during the decision-making. In order to limit my conspicuousness, I did partake in
conversation during breaks and prior to the panels. Having a working relationship with those in the observation can be thought to lower reactivity, and the trust in my professional relationship with the PRSF staff present mostly allowed me to quietly write without having attention drawn to me.
Above, I stated that unobtrusive observations were suited to case study research. One prime reason for this is the expectation that unobtrusive
observations are one of several methods a researcher might engage in during research. Lee shows that some of the first users of unobtrusive observations, such as Webb et al. (1966 cited in Lee, 2000), saw a particular need for the use of multiple methods of data collection to correct any issues created by participant reactivity to the researcher (Lee, 2000: 8). Case studies also often employ multiple types of collection methods. Unobtrusive observations of Momentum assessment panels were used in this research, together with the four other types of collection methods described above. The aims of employing and collecting the different data
76
types was to both capture different views of value from participants, and to triangulate and verify the findings between participants.
The style of observation used in the Momentum panels is also important.
Simple observation is “focused on situations in which the observer has no control over the behaviour or sign in question, and plays an unobserved, passive and nonintrusive role in the research situation” (Webb et al. 1966: 112 cited in Lee, 2000: 33). Simple observation is different from observation that uses video or film coding because it happens “contemporaneously” (McCall, 1984: 265 cited in Lee, 2000: 33). Important for the data collection process, simple observation also has
“systematicity,” meaning there is a more systematic approach to the note taking and of what is observed (Lee, 2000: 33). Lee summarises the Webb et al. (1966) approach to simple observation into the five topics:
1 exterior physical signs 2 expressive movement 3 physical location
4 what might be called ‘in situ conversation’, which involves the recording of randomly selected overheard conversations
5 behaviour associated with time (Lee, 2000: 34) Let us consider each one:
(1) In the Momentum panel observations, the exterior physical signs of the advisors – in terms of body language and verbal interactions – and the staff in the room were noted when possible, but emphasis was not placed on physical appearance as much as verbal and physical interaction.
(2) Expressive movement was useful in observing the ways advisors interacted with the music samples that were played, and notes about the body
language and motions of advisors were noted when possible. Nonverbal cues were not noted or analysed in depth, but consideration was given to the ways and frequency that certain advisors spoke, considering whether they were likely to interrupt or talk over other advisors. For the panels, advisors bring their top eight choices into the meeting. I was interested in whether those who spoke more had more success than those who spoke less with having their top choices funded.
77
(3) Physical location was noted at times, especially if the environment seemed to intervene, as when the room the panel was in became so hot it solicited unanimous comment. Such acknowledgement of the oppressive
temperature by the panel might derail discussion or reinforce a camaraderie between advisors.
(4) As the panels are limited by time, lasting approximately four hours, and because they only occur once with the same advisors present, in situ conversations were noted as observed but these were limited in number. In some ways, the time-restricted nature of the Momentum panels shaped the observations and what was observed to include time-related behaviour.
Time was a consideration because final funding decisions need to be made within the allotted time and negotiating during the last half to hour of the panel is driven by this then-urgent need.
(5) Behaviour associated with time was not specifically noted in length, as in how long something occurred, but notice was made of whether a particular person spoke longer or more often. Time spent in the panels was noted if it seemed to affect the need to make decisions more quickly, say if the panel was running over time.
Overall, effort was made in observations to minimise reactivity to my presence whilst capturing detail of the situations and decision-making process as it unfolded between advisors. Overlapping the observations, I began conducting in-depth interviews with artist managers and funded musicians, which I will now describe.