Concurrent bimodal visual and tactile stimuli were presented in the participant‟s left visual field and to the left index finger respectively (Figure 5.1). Each stimulus consisted of two intrinsic features: shape (horizontal or vertical) and frequency (low or high). The visual stimulus was displayed on a 4 × 5 array of LED light-points, geometrically analogous to the 4x5 array of pins of the tactile stimulator. The visual stimulus was located in the periphery visual field, ~8o horizontally from the fixation point. The size of the visual stimulus was ~0.5o (W) × ~0.5o (H) and the target was displayed in pure green colour.
The multisensory feature attention task was conceived as a 2 × 2 factorial design, including two levels of attended modality (vision or touch) and two levels of attended feature dimension (shape or frequency). At the beginning of each block, participants were presented with a small visual cue that appeared in the right visual field, informing the combination of sensory modality and feature dimension that was task-relevant (and, thus, to be attended) during the upcoming block. Four possible cue instructions were presented using a static or flickering (4Hz) symbolic pattern resembling character „T‟ or „V‟, which denoted the tactile or visual modality, respectively. A non-flickering character denoted attention to the shape dimension whereas a flickering one denoted attention to the frequency dimension. In combination, therefore, a non-flickering character „T‟ was displayed for the instruction to attend to tactile-
100
shape, a flickering character „T‟ for instruction to attend to tactile-frequency, a non-flickering character „V‟ for attention to visual-shape, and a flickering „V‟ for attention to visual- frequency. The cue was presented for 1s. After cue presentation, there was a 6s rest period before the onset of the first trial. Each sustained attention block consisted of 8 individual trials. The inter-trial intervals (ITI) were randomly jittered with the mean values of 4s (ranging from 3s to 7s).
On each trial, stimuli were delivered concurrently in both modalities for 400ms. Depending on the instruction given in the beginning of each block, after target presentation, participants were asked to identify either the shape (horizontal/vertical) or the frequency (low/high) of the stimulus in the attended modality. A non-speeded response was collected within 2s period after the target offset. The first response button was assigned to the horizontal or low frequency response and the second button for the vertical or high frequency one. This response mapping was reversed in the second half of the experiment to avoid correlating motor effectors with feature dimensions in the MVPA decoding of the attended modality.
Prior to the scanning, participants were familiarized with the multisensory feature attention task in a mock scanner for one session (two hours) using the same apparatus as the actual scanning. During this training session, the task difficulty was regulated either by adjusting the contrast of the shape or the difference in the frequency, so that the participant achieved ~80% correct performance on all four attentional conditions.
5.2.4 MRI Data Acquisition
Imaging data were acquired on a General Electric 3T scanner at the CUBRIC. Functional images were obtained using a T2-weighted echo-planar imaging (EPI) sequence with a repetition time (TR) of 3000ms, echo time (TE) of 35ms, flip angle of 90o, matrix size of 64 × 64, 46 contiguous slices covering the entire brain, and a voxel resolution of 3.2 × 3.2 × 3.2 mm. For each experimental run, 116 volumes were acquired, lasting 5.8 min. The scanning was conducted in a single 2-hour scanning session, consisting of 8 experimental runs with 8 blocks per run (64 trials), and a total of 512 trials.
In a separate scanning session, a whole-brain structural image, a retinotopic mapping functional localizer, and a finger-somatotopy localizer were acquired. T1-weighted structural scans were obtained with 1 × 1 × 1 mm voxel resolution, repetition time (TR) of 8ms, echo
101
time (TE) of 3ms, flip angle of 20o, matrix size of 256 × 256, and 172 slices. Functional images for the retinotopic mapping analysis were obtained using a T2-weighted EPI sequence, with a TR of 3000ms, TE of 35ms, flip angle of 90o, matrix size of 128 × 128, 37 slices, and a voxel resolution of 2 × 2 × 2 mm. Two runs of retinotopic data were acquired (one each in a clockwise and anticlockwise direction), each consisting of 100 volumes, lasting 5 min.
The finger somatotopy functional localizer was obtained using a T2-weighted EPI sequence, with a TR of 3000ms, TE of 35ms, flip angle of 90o, matrix size of 64 × 64, 46 contiguous slices, and an isotropic voxel resolution of 3.2 mm. Two runs of a somatosensory functional localizer were acquired, each lasting 5 min (100 volumes). These localizers were used to define the ROIs in the primary somatosensory cortex for each participant.
5.2.5 Retinotopic Mapping
The same retinotopic mapping task and analysis procedure described in Section 4.2.6 was used to define the regions of interests (ROIs) of visual areas V1, V2, V3, V4, V5, V7, and V8 in each subject. The retinotopic mapping data from 2 subjects were excluded due to excessive head movements during the scanning, hence data from 11 subjects were analysed in the subsequent ROI-based analysis.
5.2.6 Eye Tracking and Finger Pressure Analysis
During the scanning session, fixations of the eye, pupil diameter, and downward pressure of the left index finger were recorded in real-time and saved as a time series data. For each trial, we summarized these time series by averaging the values within the window of 100ms before the onset and 500ms after stimulus presentation, resulting in three measurements per trial (a three-dimensional vector), consisting of average horizontal eye position, average pupil diameter, and average downward pressure. Similar to experiment 4 reported in chapter 4, we used linear-SVM classification to test whether there is any pattern of information in the eye movements and finger pressure that could distinguish the currently attended modality. Classification was performed using the same leave-one-run-out cross-validation test that was used to classify the imaging data, and the average classification accuracy was obtained by averaging 8 separate training and testing iterations. Statistical significance of the classification
102
result (p-values) was estimated by running the same classifier with randomly permuted class labels for 1000 times. We then calculated the number of random samples that classified above the values of the original classification result, yielding the nonparametric null distribution of the sample.