Research ARGs provide a number of opportunities for improved data col- lection. Increased complexity over lab research combines with the increased controllability over field research to produce novel advantages.
ARGs make it possible to collect detailed information about a wide set of variables. With player permission during informed consent, and by fo- cusing on key venues where interaction takes place, researchers can record and analyze information about conversational content, tone of voice, body and language in near-real-world environments. Montola, Stenros, & Waern (2009) discuss some of these possibilities, and their limitations: • Cameras and microphones can record player activity—however, this becomes more difficult if players are moving around, and often pro- duces poor-quality records of multi-person social interaction. There are also legal limitations on recording in public spaces, even with player consent.
• Dedicated devices or programs for player use can record log files on technology-mediated communications and actions.
• Player self-reports can be requested either as part of the game or as a separate activity. The Babylon system (Waern, Ahmet, & Sundstrom, 2009), for example, provides a graphic interface where players can log both their actions and emotional responses. While it improves the quantity and depth of data, participants also report some disruption to their play experience. Reporting within the context of the story is less disruptive to play, but carries its own risks—for example, players will tend to “narrativize” their reports of events, telling an entertaining story
rather than giving a careful and accurate summary.
• Online player forums are a long-term record of not only specific social interactions, but players’ plans and interpretations of game materials. These interactions are often on a ‘meta’ level, with players speculating about, and trying to second guess, puppetmaster intentions.
Informants and game-runners can also embed themselves in player groups, providing detailed information about player interactions. Sten- ros, Waern, and Montola (2012) discuss this possibility in greater detail. They emphasize that while not every recording and analysis method must involve this kind of participation, participatory methods are a vital component of any analytic plan. Including researchers in the process of play ensures that records will be understood in the context of player expe- riences and player-created culture. For example, embedded researchers may be able to pick up on subtle cues as players shift in and out of the game’s in-story framework while discussing strategy. As described above, the pattern of these shifts may vary between games, or between players of different backgrounds.
Analysts working with the I Love Bees player forum engaged in extensive content coding, analyzing language use within each user-created discus- sion thread (Gurzick et al., 2011). This type of analysis draws on stan- dard discourse analytics methods in which researchers categorize semantic and other content, but extends these methods to understand exchanges in the context of game strategies and the emergence of the ARG’s sto- ryline. Gurzick et al. focused in particular on the emergence of collab- orative problem solving, using text analysis to explore the dynamics as players created, organized, and dissolved spontaneous teams in response to in-game needs.
Network analysis is another method well-suited to ARG-produced data. The organizers of Reality Ends Here studied the formation of creative teams, and connections across teams, over the course of the game (Stokes,
Watson, Fullerton, & Wiscombe, 2013). This game was deliberately designed to foster collaborative creation of multimedia art, and is a good example of how ARGs can be used to both produce and study complex, ecologically valid social interactions. Results supported and extended pre- vious findings in network analysis—students with greater network cen- trality, who acted as creative hubs, also had higher game scores (reflecting greater productivity and creativity).
A key advantage of ARGs in this arena is their multi-modal presentation and interaction across media and settings. These cross-modal communi- cations and relationships can provide the type of multidirectional input that is characteristic of real world events and people. However, Reality
Ends Here illustrates the advantage of central reporting for these complex
interactions—players used the full range of tools available to them for their collaborations, but ultimately brought their artwork to a central office for scoring.
ARG data collection faces 3 key challenges:
1. Some types of data collection can be intrusive, interfering with the eco- logical validity of the scenario. Self-reporting, if it takes place outside of the story, may distract from, and thereby change, in-game interactions. Players’ awareness of surveillance may also affect interactions and lead them to deliberately “perform for the camera,” while a lack of aware- ness has serious legal repercussions. Players may try to identify or fool embedded researchers as part of their strategy.
2. As described above, players reporting on their own experiences may attempt to revise them into game-appropriate narratives, increasing reported drama or leaving out details that don’t seem to fit. They may also treat manipulation of in-story data collection as part of game play—for example, providing false answers to a survey in order to try and reach an in-game goal.
3. Collection during complex interactions can be logistically, and sometimes legally and ethically, difficult. Face to face interactions are of particular interest for ecological validity, but may be difficult to record if they don’t take place under carefully controlled circum- stances. Cross-modal discussions may be difficult to follow, and key components may be missed as players seek new tools and venues for reaching their goals.
Even during non-research games, puppetmasters encounter these issues. Experienced game-runners report a variety of compromises. Carefully discrete embedded observation is generally agreed to be necessary. Even participants who have signed informed consent agreeing to surveillance may not pay it much attention during actual play, particularly if the story is diverting enough. Montola, Stenros, and Waern (2009) recommend having a story-based explanation available if these things are discovered during play sessions. (For example, a hidden camera might include a clue pointing to the identity of adversaries.)
Ultimately, while all methods of recording data during ARG play have their limitations, a well-chosen selection of complementary strategies can provide parallax.