5.4.1 Variables
The variables for this experiment are inherited from theSVPexperiment discussed in chapter4.
This identicaltargetidentification task ulitises six strips of terrain, referred to as terrains A–F, each featuring three clusters oftargets, with each cluster containing 1–3 human beings — an accurate representation of thetarget stimulifor aWiSARtask. Unlike the previousSVPexperiment, there is only a single flight speed of 90 mph (145 km/h). Therefore the dependent factor in this experiment issegmentationdegree— of which there are six as previously described— labelled 1–6. Hence there are 36 combinations of terrain andsegmentationdegree to be tested (i.e. an experiment block), as can be seen in table5.3. This results in 108 individual identification results (targetfound or missed) per experiment block.
As before, several limitations exist to minimise any potential learning effects: the same terrain orsegmentationdegree cannot be shown to an individual subject more than once. Such limitations correspond to a maximum of six experiment trials per subject. Conflated with the fixed flight speed
Table 5.3– A breakdown of all the variables and their respective counts within the segmentation experiment
6 Segmentation Degrees 6 Terrain Strips
6 Experiment Trials per Subject 3 Clusters of Targets per Terrain Strip
108 Identification Results per Experiment Block
of 90 mph, and a terrain strip length of 1.45 miles (2.34 km), the result is that each experiment trial will last approximately 58 s1.
The standard method of allocating variable combinations to subjects in a fair method to avoid systematic effects is aBBD, as explained and used previously in the design of theSVPexperiment in chapter4. Due to the time limit of six experiment trials per subject, and 36 combinations of terrain andsegmentationdegree, an experiment block is defined as 36÷ 6 = 6 trials. Herein lies a problem: a randomised balanced block design has yet to be discovered for a 6× 6 design. This is also known as the 36-officer problem, and no known solution exists (Klyve and Stemkoski 2006). The nearest solution is one where a pair of experiment blocks must be rotated. Therefore for all 36 combinations of terrain andsegmentationto be tested, 12 subjects each performing six trials at the six degrees ofsegmentationmust be conducted to achieve a balanced design yielding two experiment blocks, or 216 identification results (i.e. two trials per individual combination of terrain andsegmentationdegree).
5.4.2 Experimental Procedure
The experiment was strictly controlled, with a single subject sat in front of thestimulimonitor beside the investigator who controlled the flow of the experiment and dealt with any eye-gaze tracking issues. To reduce the workload on the experimenter and to ensure consistency, an introduction video was developed with an improved version of the presentation software. This was in addition to the usual consent forms and experiment scripts.
1 Sequence durations cannot be provided exactly due to the use of tile jittering explained in section5.3.2.
Introduction Video
To maintain identical conditions between subjects, an introduction video was developed to explain the experiment and to provide a minimal amount of training for volunteers. Using a video instead of a script (as used in theSVPexperiment) eliminates any differences in emphasis or differences in timing that could impact the way in which subjects act.
The silent, subtitled video lasted one minute and 31 seconds and was split into four sections:
1. Introduction: This brief section introduced the research study and reiterated the key points from the ethics statement.
2. WiSAR: The second section included video footage of a UAV and explained their role in SAR.
3. Training: The third section included a still image of the sample card (figure5.3) along with an explanation of how many space-bar presses were expected from the subject.
4. Experiment Issues: The final section explained the experiment procedure and explained the eye-gaze camera in front of the subject.
Careful phrasing was used during the instructional part of the video to ensure that subjects knew when to press the space-bar key, and also to manage any expectations subject’s may have regarding the number oftargetsper sequence. Participants were told to “press the space-bar immediately whenever you see a human being”, and that “there may be none, one or more human-beings in each sequence”. Figure5.3, the sample card, was shown to participants, complete with the circles and numbers to indicate the varied appearance of the human-beingtargets. In addition to two individualtargetsat the top of the sample card there is a cluster of twotargetsat the bottom-left of figure5.3. The accompanying subtitles explain that should this sample card be shown, four space-bar presses would be expected in quick succession.
Presentation Software
After displaying the introduction video to the subject, the system performed a standard eye-gaze tracker calibration routine as described in chapter3. With the eye-gaze tracker successfully
2 1
1
Figure 5.3– Example frame indicating four sample target, used in to train the subjects to look for clustered targets
working, the six sets ofstimulitiles at the requiredsegmentationdegrees were loaded into memory for rapid retrieval. The software controlled the sequence of stages within the experiment, as shown in figure5.4. Once ready, the software indicated this fact to both the subject and investigator. The investigator confirmed that the eye-gaze tracker was successfully sending eye-movement data from their terminal before verbally asking the subject if they were ready (shown as diamonds in figure5.4). From here the investigator pressed a key for the experiment to proceed.
Statements
Title Introduction Calibration
Billboard Central Gaze
Attractor Trial
Sequence
Loading
Screen Comparisons Thank You
Start
End Introduction and Training
Experiment Trial
Debrief
Figure 5.4– The visual computer-controlled procedure for thesegmentationexperiment, demonstrating the three stages presented to a subject: introduction, experiment trials and the debrief, for the successful completion of six experiment trials. Diamonds indicate interaction performed by the investigator to switch displays or advance the presentation.
From the perspective of a subject, a central “gaze attractor”— a pulsating white circle shown to encourage the subject to foveate thestimuli— was presented to centre the gaze position on screen immediately prior to thestimulibeing presented. After two seconds the experiment trial began with the predefined terrain strip shown through a specificsegmentationdegree. At the end of each trial, the software presented the survey, or questionnaire to the user.
Subjective Questionnaire
Following each individual experiment trial, subjects were asked to give a self-assessment of their performance by indicating their agreement or otherwise to four statements covering target identification confidence, task challenge, image size and presentation rate. These questions were chosen to identify perceived differences intargetidentification difficulty, as well as to gather opinions on the rates and dimensions of the presentation imposed by the experiment design.
These four statements were presented on the screen sequentially (figure5.5) and advanced by the investigator. Responses were verbal and recorded by the investigator on a paper form as Likert scale values for the purpose of analysis.
At the conclusion of all six presentations each subject was then presented with an overview of the sequences they were shown, in the order in which they were shown. Subjects were first asked to choose which sequence they were most successful using for the task, and then which presentation mode they preferred. Figure5.6includes an example trial with the two comparative questions. Again the experimenter recorded the verbal responses. While the answers for these two questions are likely to be similar (i.e. on the supposed principle that individuals tend to prefer things that assist them), the choice of two comparison questions was used to evaluate whether the segmentationdegree was suitable for thevisual searchtask, and also whether the subject could endure observing a particularsegmentationdegree that potentially maximises their chances of success.
5.4.3 Conducting the Experiment
Twelve subjects were initially recruited for thissegmentationexperiment, none of whom particip-ated in the previousSVPverses Moving experiment. All were drawn from the student and staff
(a)Target Identification Confidence (b)Task Challenge
(c)Image Size (d)Presentation Rate
Figure 5.5– The four post-presentation subjective questionnaire statements, shown sequentially to the subject after each experiment trial. These statements were answered verbally and advanced by the investigator.
(a)Success (b)Preference
Figure 5.6– The two post-experiment comparison questionnaire statements, shown with a live preview of the trials observed (arbitrarily named A–F) during the experiment, in the order in which they were shown to the subject.
population of the Electrical and Electronic Engineering department of Imperial College London.
The average age of the group was 26.5 years, and all signed informed consent forms. Due to the presence of rapidly changing visuals (i.e.segmentationdegree 6), subjects were asked to confirm that they were not sensive to rapidly flashing visuals at a rate of up to 10 Hz, in addition to any uncorrected vision deficiences (e.g. shortsightedness).
The experiment was conducted in an identical fashion to all the investigations described in this thesis, with the investigator sat to the side of the subject. As before a ny comments made by the subject were recorded on paper for further analysis.