Method 1. Pilot Study

second half of the sample. Third, we analysed the convergent validity of the scale by looking at the relationships with other variables that were expected to relate to digital skills. Accordingly, a review of the literature shows that there is a positive correlation between high levels of digital literacy and a favourable attitude towards technology (Ainley, Fraillon, Gebhardt, & Schulz, 2012; Zylka, Christoph, Kroehne, Hartig, &

Goldhammer, 2015), and a negative association with anxiety towards technology (De Wit, Heerwegh, & Verhoeven, 2014; Durndell & Haag, 2002; Zylka et al., 2015).

Similarly, age is expected to relate to the level of digital literacy, with older students having a higher level than younger students (Appel, 2012; Kuhlemeier & Hemker, 2007;

Mascheroni & Ólafsson, 2014; Sonck et al., 2011). Likewise, male adolescents tend to report a higher level of digital literacy than female adolescents (Hakkarainen et al., 2000; Kopaiboon, Reungtrakul, & Wongwanich, 2014; Kuhlemeier & Hemker, 2007;

Mascheroni & Ólafsson, 2014; Y. J. Park, 2013).

6.2. Method

Table 16.

Items excluded because of extreme values of skewness and kurtosis

Items (I know…) Skewness Kurtosis

How to download apps to my mobile or tablet without help -3.23 10.51 How to uninstall apps when not liked or needed -4.18 16.88 How to go to a previous page when browsing the Internet -2.00 3.26 How to upload images, videos or music onto any of my social

media profiles (e.g., Twitter, Facebook, etc.) -2.11 3.38 How to use different tools on the mobile phone to

communicate with others (WhatsApp, email, phone call,

etc) -2.02 3.52

Which information I should and shouldn’t share online -4.39 22.09 Things on the Internet disappeared with time, so it does not

matter what you post on the Internet -2.33 4.91

How to block messages on social media sites from people

you do not want to be in contact with 2.72 6.65

How to block messages on instant messaging program from

people you do not want to be in contact with -2.04 3.27 I share my social media passwords with other people -2.15 3.62 How to protect a smartphone with a PIN or with a screen

pattern -3.72 13.30

6.2.2. Main Study: Sample and Procedure

From March to May 2016 the survey was conducted. Data were collected from 1,467 students of secondary education from 1^st to 4^th grade of secondary education at thirteen schools in both rural (seven schools) and urban (six schools) areas of Spain (51% respondents from a rural school). Among these thirteen schools, four were public, eight (publicly funded) private and one private (see Appendix 1). Students were between 12 and 18 years old, and their mean age was 13.97 (SD = 1.31). Moreover, 52%

of participants were men and 48% were women. Of the 1,467 participants, 21 were excluded from the analysis because they did not properly fill out the questionnaires.

Therefore, 1% of records were incomplete and considered invalid. In the end, 1,446 participants were retained for the analysis.

The thirteen participating schools (and the participating school in the pilot study) were self-selected, responding to an invitation to participate in the research project (see Appendix 2). The online invitation to participate in the experiment was sent to 327 schools. The contact rate was 7.64%, since only 25 attempts to contact were successful, but then some schools refused to participate for different reasons (e.g., lack of time). Finally, the response rate was 3.66% as 13 of the contacted schools agreed to participate.

During the survey, participants were administered a paper-and-pencil questionnaire under the supervision of a researcher and teacher during class. The survey took about 15-30 min to complete and it contained measures of digital literacy, positive ICT attitude and technology anxiety (as convergent validation measures), as well as other variables not of interest for the current study.

A consent form was sent to parents/carers of students in the participating schools (see Appendix 3). Participants were told about the nature of the study and given contact detail of the study’s representatives to consult if necessary. They were guaranteed confidentiality and anonymity. Informed consent was received from school staff (school principals and heads of studies) and the corresponding Provincial Education Directorate. Participants did not get any compensation for taking part in the study.

6.2.3. Measures

Digital Literacy: The digital literacy scale consisted of 35 items that were measured with a 5-point Likert scale of self-reported agreement, responses ranging from 1 = strongly disagree to 5 = strongly agree. These items were adapted and created for each of the previous five dimensions identified for digital literacy identified (five digital

skills: technological skill, security skill, critical skill, informational skill and communication skill).

Convergent validity: To assess convergent validity we included two measures that were expected to relate to digital literacy: positive ICT attitude and technology anxiety.

Positive ICT attitude was assessed using eleven items adapted from the literature (Fraillon, Ainley, Schulz, Friedman, & Gebhardt, 2014; Ng, 2012; Wilkinson, Roberts, & While, 2010). Items were measured with a 5-point Likert scale of agreement, from 1 = strongly disagree to 5 = strongly agree (e.g., “It is more fun to do homework using a computer than without it”). All items loaded on the one factor, with factor loadings ranging from .40 to .77 and an explained variance of 39% (α=.84;

M =3.57, SD =.71).

Technology anxiety was measured with nine items on a 5-point Likert scale of agreement, from 1 = strongly disagree to 5 = strongly agree, which have been used to assess technology anxiety in previous studies (Barbeite & Weiss, 2004; Heinssen, Glass, & Knight, 1987). The nine items (e.g., “Computers make me feel uneasy and confused”) loaded on one factor with factor loadings ranging from .58 to .73 and an explained variance of 43% (α=.83; M =1.89, SD =.68).

6.2.4. Data Analysis

Data were analysed using SPSS 22 and AMOS 20. First, and to examine cross- population validity, we randomly divided the data into two groups (Sumter et al., 2015) i.e., sample 1, n = 715 and sample 2, n = 730. The first sample was used for conducting exploratory factor analysis (EFA) to discover the latent digital skills. The internal consistency of these constructs was then checked with Cronbach’s alpha. The convergent validity of the Digital Literacy Scale with the Positive Attitude towards Technology Scale and the Technology Anxiety scale was assessed using Pearson's

correlation coefficient. Subsequently, the second sample was used for conducting confirmatory factor analysis (CFA) on the Digital Literacy Scale with AMOS.

Given the fact that the Shapiro-Wilk test showed that the variables failed to meet the normality assumption, a bootstrap method was used in the analysis with AMOS.

We estimated 95% bias-corrected confidence intervals with 1.000 bootstrapping samples. In this context, an association is considered statistically significant if the confidence intervals (95% BCI) do not include zero. Taking into consideration that a bootstrap method cannot be performed with missing data, missing values were replaced using the linear trend at point technique in SPSS. None of the variables had more than 3% cases missing and most of them had missing data in less than 1% of the cases. Results were not affected neither by the replacement missing values nor by the bootstrap methods.