Participants
Recruitment was conducted via advertisements at Cardiff University and Prolific2 (https://www.prolific.ac/) and data were collected in both laboratory and online settings (see section 4.3.1). 254 individuals were recruited via Prolific and 18 drop-outs were recorded. There were 30 recruited individuals who were not eligible to participate and quit the study (see section 4.5.1). In laboratory settings, a total of 114 participants was recruited. 43 participants recruited via the Experimental Management System (EMS) received course credits when eligible (e.g., undergraduate students) and 71 participants not eligible for course credits received monetary reimbursement (£6). Participants performing the study via Prolific were rewarded £4.50 upon completion3. The complete and exhaustive set of inclusion and exclusion criteria for participation in the study were as follows. Eligible participants were at least 18 years of age, had normal or corrected-to-normal vision, including normal colour vision, and spoke English as their first or second language. Exclusion criteria included being on a diet and/or have recently been taking diet pills, a past and/or current history of eating disorders and food allergies and/or intolerances. Screening survey questions can be found in Appendix G and all criteria were based on self-report. Further post-hoc exclusions of participants from preregistered analyses are explained in section 4.4.3 and presented in section 4.5.1.
The study was approved by the local Research Ethics Committee at the School of Psychology, Cardiff University. All eligible participants provided informed consent and were debriefed. The study employed a within-subjects design and blinding of participants and/or experimenters was not applicable. However, participants were not made aware of the study aims before completion. Also, the contact between the experimenter and participants was minimised as data was collected online and in group laboratory settings, as explained in the next section.
2Prolific requires pre-screening of participants and the current country of residence was set to UK
for two reasons: 1) consistency of subject pools between laboratory and online settings; and 2) food brands included in the behavioural tasks might not have been popular outside the UK.
3A £6.00/h rate was used for both Prolific and EMS participants. On Prolific, the estimated time
of completion was 45 minutes, and, in the laboratory, the study was expected to last 60 minutes due to the coordination of group testing settings and the time it would take to provide all participants with the chosen food items (see section 4.3.2).
Study setting
The study was undertaken in both laboratory (group testing) and online settings using Inquisit 5 (Millisecond Software, 2017). The study protocol was matched for the two collected data sets, which were analysed and reported separately. The primary dataset stemmed from the laboratory setting, as this would allow us to examine consequential food choices (see section 4.3.4). The online dataset would directly replicate any findings on the APP as an implicit measure of food attitudes (H1-H3) and examine whether priming measures were associated with non-consequential food choices (i.e., choices are not motivated by the offer of real food at the end of the experiment). This data collection protocol would also provide insights into data quality and potential differences in the utility of the APP between laboratory and online settings (see section 4.5.4).
Sampling plan
A Sequential Bayes Factor (SBF) design with maximal N (Schönbrodt et al., 2017) was employed, whereby data collection continued until either the desired level of evidence for all confirmatory hypotheses was obtained within each study setting separately (online and laboratory), or the maximal N had been reached. A minimum sample size, or nmin, of 40 was set for each study setting and the maximum number of participants, or nmax, was 2004. A threshold of BF10 ≥ 10 would indicate strong evidence for the
alternative hypothesis (H1) compared with the null (H0) while threshold of BF01 ≥ 10
would correspond to strong evidence for H0 relative to H1 (see Lee & Wagenmakers, 2013). Interim analyses were conducted for every 10 participants. The evidential value and hence interpretation of the results was exclusively based on Bayes factors, but frequentist statistics have also been reported (α = 0.05). Although frequentist power analysis was not appropriate for an SBF design, a Bayes Factor Design Analysis (BFDA; see Figure 4.2) was conducted to assess the probability of the proposed design generating misleading evidence (Schönbrodt & Wagenmakers, 2018). Analyses were performed for all preregistered hypotheses, as in directional t-tests for priming-related hypotheses (H1-H3) and directional correlations for food choice task predictions (H4), as shown in panels A and B of Figure 4.2, respectively. The design priors were consistent with the analysis priors that would be employed for Bayesian t-tests and
4For the purposes of the present thesis, I need to report that nmax has not yet been reached
for the laboratory cohort due to limited time and availability of participants at Cardiff University during the preceding months of final thesis submission. Data collection has been set to continue after a specified date: 05/11/2019.
correlations (see section 4.4.2). Only the BFDA results were considered for the design of the study and no other power analyses were conducted.
Figure 4.2: Bayes factor design analysis (BFDA) results for H0 and H1 at different sample and effect sizes in a simulated sequential design. The graphs show the percentage of
simulated studies (10000) terminating at a boundary (H1 and H0) when the threshold is set to BF01
≥ 10 and BF10≥ 10. BFDA has been conducted for the sample sizes of 40 (nmin), 80, 120, 160 and
200 (nmax). The Cohen’s dz and correlation ρ (rho) values reflect the potential of the true effect size
being either zero (i.e., H0 is true), ‘small’ (dz = 0.2; ρ = 0.1), ‘medium’ (dz = 0.5; ρ = 0.3) or ‘large’
(dz = 0.8; ρ = 0.5). These benchmarks are used for demonstration purposes only. A. This panel
shows the BFDA results for the planned directional Bayesian paired-samples t-tests (H1-H3). For H0, 77.04% of all simulated studies correctly terminate at the H0 boundary when
nmax has been reached. However, the probability of obtaining false positive evidence is low, with
only 1.5% of the studies incorrectly stopping at the H1 boundary. At n = 40, the probability is very low, with only 0.36% of studies incorrectly stopping at the H1 boundary.
Assuming a small true effect size for H1 (dz = 0.2), at nmax 57.7% of simulated studies terminate at the correct H1 boundary and 5.9% of studies stop at the H0 boundary (i.e., probability of obtaining false negative evidence). Assuming a medium true effect size (dz = 0.5), at a sample size of 120, 99.7% of all studies correctly terminate at the H1 boundary and no studies (0%) stop at the H0 boundary. For a large true effect size (dz = 0.8), 80 participants would be adequate to correctly support H1 with 100% of simulated studies correctly reaching the H1 boundary. B. This panel shows the BFDA
results for the planned directional Bayesian correlations (H4). Assuming the absence of a
positive correlation (H0) at nmax, 71.19% of all simulated studies correctly terminate at H0 and 1.92% incorrectly stop at the H1 boundary. For a small true effect size (ρ = 0.1), only 14.85% of studies correctly terminate at H1 when nmax is reached and 25.54% of studies stop at H0. For a medium true effect size (ρ = 0.3), at nmax 95.65% of all studies correctly provide strong evidence for H1 and for a large effect size (ρ = 0.5) at the sample size 120, 99.94% of all simulated studies correctly terminate at the H1 boundary. Note. The design priors were consistent with the planned analysis priors (see section 4.4).
4.3.2
Procedure
Recruited participants confirmed their eligibility and proceeded to provide their consent and choose their study setting (laboratory or online). Participants also indicated their dominant, or preferred, hand for performing the study tasks. A schematic of the study procedure is shown in Figure 4.3. The prime selection process required participants to complete a rating task where they rated how much they like food and non-food stimuli (see section 4.3.3). Participants completed a short APP practice block (16 trials), where they received feedback on both the speed and accuracy of their responses. Participants completed eight blocks of the task in total, with short breaks in between and instruction reminders.
After the APP, participants performed a food choice task (FCT; see section 4.3.4), consisting of two blocks in total. In laboratory settings, participants received a food item chosen during the task at the end of the study. In online settings, food choice was not consequential in terms of real consumption. Ratings for all primes and targets (see section 4.3.5) were provided after the FCT for exploratory analyses. Participants were presented with three short questionnaires5 (see section 4.3.6). The total duration of the study per participant was 40-50 minutes, after which participants were debriefed.
5Questionnaire items may prime participants to pay attention to health- or weight-related
Figure 4.3: Schematic of the study procedure and affective priming paradigm. A. Primes
for the affective priming paradigm (APP) were selected based on participants’ liking ratings, with the subsequent APP consisting of eight blocks, including 32 food and 8 non-food prime trials per block. The food choice task followed the APP and included two blocks of 64 trials. Participants then rated all primes and targets and were presented with three short questionnaires in the depicted order.
B. The APP involved an evaluative categorisation task, where participants categorised target words
as positive or negative as quickly and accurately as possible. After a central fixation cross (1000ms), a prime (food or non-food) was presented for 233ms, followed by a mask. Participants must respond within 1500ms of target onset using the “G” and “H” keys for positive/negative (randomised across participants) using their index and middle fingers. Finger placement on the assigned keys depended on the participant’s dominant hand.