While Player Experience is an important topic with strong significance to this research domain, the subjective nature of its varied components warrant the need for different forms of qualitative and quantitative assessment. Identifying suitable measures of player experience will be necessary for the data collection and analysing process to have some degree of structure. It is also useful to discuss player experience measuring techniques to determine their usefulness and effectiveness outside their respective sources.
2.7.1 Talk out Loud
The qualitative technique of permitting a participant to describe aloud what they are thinking while performing a set task is a popular expository method used in studies. The origins of the technique as an approved research technique are somewhat convoluted. However, it was introduced and described for use by Lewis and Rieman in their research regarding task-centred user interface design (Lewis & Rieman, 1994). It describes the basic principles of the technique as well as some of the benefits and cons of employing it efficiently. As an example, Talk out Loud is seen to be an effective means of extracting information from participants in real- time, but it is unfortunately subject to flaws possessed by its human subjects and the task at hand. A particularly quiet participant that is difficult to understand or needs
6Dijksttra algorithm - a graph search algorithm that solves the single-source shortest path problem for
a graph with nonnegative edge path costs producing a shortest path tree.
7 A* - pronounced "A-star", in Computer Science it is an efficient and widely used best-first search
56
regular prompting may struggle with concurrently talking and performing a task. Data yielded from such an individual may be unsatisfactory in comparison to participants who have no problems.
Additionally, the research by Johnstone et al., used talk out loud to evaluate test designs, brings up the technique in detail. The paper is particularly helpful as it analyses the technique's usefulness using a demonstrative example study (Johnstone et al., 2006). This study was proven to benefit greatly from the subjective, participant based communication technique, garnering data that would otherwise have been unbeknown to the researchers.
2.7.2 PENS & GEQ Scales
The Player Experience and Needs Satisfaction (PENS) survey is developed by Immersyve (www.immersyve.com), a multidisciplinary team of researchers located in Orlanda, Florida in the United States. PENS is derived from the research of several individuals regarding motivation and satisfaction in many forms of digital media, particularly video games (Rigby & Ryan, n.d.). While it is still in development with a newer iteration of the scale in the works, it is based strongly around the concept of Self-Determination Theory (SDT), the theory which addresses the factors that either facilitate or undermine motivation, both intrinsically and extrinsically. The PENS consists of addressing three needs found to be necessary regarding our motivation and engagement in something, that being Competence, Autonomy and Relatedness. The PENS questionnaire itself addresses these needs using scales for testing in-game competence, in-game autonomy, presence and intuitive controls. These scales consist of either 3 or 5 similar statements that are rated on a Likert scale from 1 to 7. The PENS has been in popular use by researchers around the world since its inception and it is backed up by a plethora of research content, theory and documentation. Particularly its use by Ryan et al. (2006) in their own research provides a comprehensive description of SDT and how the PENS can be used to measure player motivation and satisfaction levels.
The Game Experience Questionnaire (GEQ) is developed by the Game Experience Lab at the Eindhoven University of Technology in the Netherlands. It is based on the research by an interdisciplinary group of scholars focused on exploring the domains of flow, immersion and engagement in video games (IJsselsteijn et al.,
57
2013). Although it has been trialled many times and is currently being employed by researchers around the world, the GEQ is still a work in progress. The GEQ has a modular structure that consists of 3 modules: Core Questions, Social Presence and Post-game. Typically these modules would be administered to a participant immediately after a game-session has finished. However, components of the modules are capable of being tested for individually. The core questions of the GEQ are divided into questions related to seven components: Immersion, Flow, Competence, Positive and Negative Affect, Tension and Challenge. Each of these components is divided into five similar questions. For example, the Challenge component of the core questions (used later in this research) asks participants to rate five questions on a Likert scale from 1 to 5 surrounding the topic of how challenging the encountered game play was perceived to be. How the GEQ is applied to academic research is open to interpretation.
2.7.3 Event Sampling
When observing participants during studies, it is useful for researchers to be able to employ qualitative observational techniques that do not disturb the participant. Unlike Event or Experience Sampling Methodologies, the technique of Time and Event Sampling allows a participant to perform a task unhindered by the burden of needing to record information themselves (Bushnell & Irwin, 1980). This is instead delegated to the researcher performing the study, allowing for consistency and greater reliability between datasets involving multiple participants. Time and Event Sampling is most commonly used when observing participants who are either incapable of recording data themselves (such as adolescents), or participants who are performing a task that both require high levels of concentration and permit high levels of flow. As playing video games, particularly those of the FPS genre, allow for both of these conditions to transpire, Time and Event Sampling should be suitable for studies requiring an event sampling technique. Additionally, Time and Event Sampling suggests four distinguishable guidelines for efficient use of the sampling technique. These are the following:
1. Clearly identify and then operationally define the behaviour that you want to study
58
2. Know enough about the behaviour in general that you know where and when to observe
3. Determine what kind of information you want to record
4. Make your recording sheet as easy to use as possible
Using these guidelines, it should be possible to create an observational strategy that is not only efficient during its undertaking, but can be used effectively to work out the frequency, priority, relevance and importance of various phenomena. In this way, the occurrence factor of phenomenon that are systematically observed can be analyzed qualitatively, and a semblance of a behaviour state list can be designed (Chikawa, Iwata, & Tano, 2002).