In addition to the static representation of eye movements, it was also important to capture a
dynamic representation of when and where participants looked, in the form of video recordings.
For the experiments described in Chapters 4 and 5, the Eyelink II scene camera was capable of
producing this without the requirement for an additional computer programme. This was
49 experiments described in Chapters 3 and 6, a computer monitor was used to display the
stimulus, and the eye tracker was unable to produce a high resolution video recording of both
the dynamic real time stimulus and corresponding eye movements. This limitation was
overcome by creating a MatLab ® (MathWorks ®, 2015) script with the Psychtoolbox
application (Kleiner et al., 2007). This, along with video capture software, allowed the eye
movement data to be recorded synchronously along with a real time display capture. It was
then possible to overlay the eye movements into the captured video using two methods. The
first involved using a MatLab script and the second involved video editing software. The main
procedures are summarised below. The full MatLab scripts used can be viewed in Appendix 2.
2.2.3.1 Recording eye movements and screen capture simultaneously
Before eye movements were overlain onto the corresponding video file using either the
MatLab or video editing software method, the initial stage of the procedure was to record a
video of the experimental trial and the eye movements of the observer simultaneously using an
intial MatLab script. The function initially opened a pre-experimental window which allowed
for eye tracking calibration and defined a start and stop key (see Figure 2.6. Comments are in
50 Figure 2.6. Script used to create the pre-experimental window and to define start and stop keys
The function wRect created a rectangular window on the desired display screen which was
defined by screenNumber; where screenNumber was usually 0,1 or 2 depending on the physical
set up of the displays (i.e. single monitor or dual monitors set up). The script then called for
the initialisation of the Eyelink using the inbuilt Eyelink Toolbox functions within
Psychtoolbox (Figure 2.7)
51 This script allowed MATLAB to open up the default calibration screen in the window as
defined and controlled within the Eyelink Toolbox directory of Psychtoolbox (el =
EyelinkInitDefualts (w)). EyelinkDoTrackerSetup(el) allowed for calibration and
EyelinkDoDriftCorrection(el) allowed for a drift correction which again was controlled by the
default scripts through the Eyelink Toolbox directory (Figure 2.7).
Figure 2.8 The function which executes the opening of FRAPS video capture software and the stimulus software executable file.
The next stage of the procedure was the simultaneous recording of the stimuli and eye
movements. This was accomplished through video screen capture executed by the MATLAB
script, which also executed the function to allow the eye tracking software to record the eye
movements. The screen capture was controlled by FRAPS video capture software. The function
commanded FRAPS software to open along with the stimulus programme to be presented (e.g.
Driving Simulator 2011). This was accomplished using the script outlined in Figure 2.8. The
actxserver function allowed MatLab to call the video recording software FRAPS and then after
52 Figure 2.9. Script to execute the recording of the eye movements and the screen capture simultaneously
After the waiting period, MatLab executed a script to allow the eye tracker to begin tracking
the eye movements and allowed FRAPS to record to the display screen. Importantly, there is a
line placed here which told MatLab that the F9 key has been pressed [h.SendKeys ('(F9)')]
which was the key used by FRAPS to begin recording the screen capture. This step allowed
both the eye movements and the screen to be recorded simultaneously.
Figure 2.10. The termination code to close the experiment whilst terminating the screen capture recording and eye movement recording
Finally, the complete function terminated the experiment, which in turn synchronised the
termination of both the eye movement recording and video capture recording. This was
accomplished with a simple check loop until the previously defined stop key was pressed. Upon
53 frame, terminated the video recording. This was accomplished using the example code
presented in Figure 2.10.
2.2.3.2 Overlaying the recorded eye movement and video file using a
Matlab script
This method was used in Chapter 3. However, this method was not developed by me, but
another researcher (Paul Cox, acknowledged). Thus, it is difficult to describe the process in
detail. The whole script can be viewed in Appendix 2.2. To summarise, the script took the raw
eye movement data created from the Eyelink software and recorded the x and y coordinates of
each fixation. It then used this information to draw a red coloured circle on the recorded video
file at the corresponding time frame. It went through the video on a frame-by-frame basis
continually drawing a red circle given the x and y coordinates of each fixation. It then pieced
each of these frames together to produce a new video file of the recorded drive and the eye
movements overlain.
2.2.3.3 Overlaying the recorded eye movement and video file using video
editing software
This method was used for each of the subsequent Chapters (Chapters 4, 5 & 6). The eye
movement video trace was exported to an avi file directly from the Eyelink Data Viewer
Software. This video file simply consisted of the eye movement pattern over time, presented
as a coloured circle, on a black background. This video file was imported to Adobe Premiere
Pro (Adobe Systems Software, 2014), along with the recorded video of the stimulus display.
Using this video editing software programme, the eye movement file was overlain on top of
the stimulus video. The black background of the eye movement video was filtered out by
54 could not be seen but the contrasting hue of the eye movement gaze cursor could. This left the
underlain stimulus recording with the overlain eye movement trace.
All other pieces of software or apparatus that was used in the experiments are described in
55