Recent research has placed increasing emphasis on Internet misuse and its consequences, both psychological and behavioural, among young people (Castellana, Sánchez- Carbonell, Graner, & Beranuy, 2007; Gámez-Guadix, Orue, & Calvete, 2013; Kaltiala- Heino, Lintonen, & Rimpelä, 2004; Kormas, Critselis, Janikian, Kafetzis, & Tsitsika, 2011; Viñas, 2009). Such consequences include the emergence of possible behavioural alterations, loss of control, school failure, social isolation and an increase in family confl ict (Cao & Su, 2006; Holtz & Appel, 2011; Sánchez-Martínez & Otero, 2010; Tonioni et al., 2012).
Over the past few years, there has been a proliferation of investigations that have attempted to examine rigorously the possibility of considering an addiction to new technologies, more particularly to the Internet and mobile phones (Davis, 2001;
Echeburúa, Labrador, & Becoña, 2009; Estallo, 2001; Young, 1998a). However, the consensus remains weak, exemplifi ed by the fact that such disorders are neither listed in the International Classifi cation of Diseases (ICD-10) nor in the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), although Internet Gaming Disorder has indeed been included in DSM-5 Section III, as a condition warranting more clinical research and experience (American Psychiatric Association, 2013). Nevertheless, these criteria are currently limited to Internet gaming and do not include general use of the Internet, online gambling, or social media.
Despite the inconsistent recognition of possible Internet addiction as a specifi c disorder, its importance as a separate entity is supported by an increasing number of studies, sparking a degree of social alarm. Durkee et al. (2012), based on a sample of 11956 young people from 11 different European countries, reported a prevalence of Maladaptive Internet Use (MIU) among adolescents of 13.5%, and estimated the rate of Pathological Internet Use as 4.4%, this percentage being higher in boys compared to girls (5.2 versus 3.8%, respectively). For its part, EU Kids Online II survey (a project developed by the European Commission on a sample of
Evaluation and early detection of problematic Internet use in adolescents
Patricia Gómez Salgado, Antonio Rial Boubeta, Teresa Braña Tobío, Jesús Varela Mallou
and Carmen Barreiro Couto
Universidad de Santiago de Compostela
Abstract
Background: Problematic Internet use in adolescents has become an issue of concern for a growing number of researchers and institutions over the past years. Behavioural problems, social isolation, school failure and family problems are some of the consequences of psychological and behavioural impact on teenagers. Taking into account the interest that this issue has generated at many levels, the aim of this paper is to develop a screening tool for early detection of problematic Internet use in teenagers.
Method: A survey of Compulsory Secondary School students from Galicia involving a total of 2,339 individuals was carried out. Results:
The results obtained allow (1) gauging the magnitude of the problem, establishing the risk levels among the adolescents, and (2) presenting a new, simple and short screening instrument. Conclusions: The present scale has suffi cient theoretical and empirical support, including good psychometric properties (α = .83; specifi city = .81; sensitivity = .80; ROC curve = .90), making it an interesting applied tool.
Keywords: Screening, early detection, evaluation, Internet, adolescents.
Resumen
Evaluación y detección precoz del uso problemático de Internet entre adolescentes. Antecedentes: el uso problemático de Internet entre los adolescentes preocupa cada vez más a investigadores e instituciones. Problemas de conducta, aislamiento social, fracaso escolar y problemas familiares son algunas de las consecuencias del impacto a nivel psicológico y conductual que ello produce. Habida cuenta del interés que el tema suscita a diferentes niveles, el objetivo de este trabajo es desarrollar una herramienta de screening para la detección precoz de uso problemático de Internet entre adolescentes. Método: se realizó una encuesta a escolares de Enseñanza Secundaria Obligatoria de la comunidad gallega, en la que participaron un total de 2.339 individuos. Resultados: los resultados obtenidos permiten: (1) evaluar la magnitud del problema, permitiendo conocer los niveles de riesgo existente, y (2) presentar un nuevo instrumento de screening o cribado, breve y sencillo. Conclusiones: la presente escala cuenta con sufi ciente aval teórico y empírico y con unas buenas propiedades psicométricas (α = ,83; especifi cidad = ,81; sensibilidad = ,80; Curva COR = ,90), lo cual la convierte en una herramienta de interés a nivel aplicado.
Palabras clave: cribado, detección precoz, evaluación, Internet, adoles-centes.
Received: April 17, 2013 • Accepted: September 27, 2013 Corresponding author: Antonio Rial Boubeta
Facultad de Psicología
25142 children from 25 European Countries) has shown that 30% of 11-16 year-old-children have experienced one or more forms of excessive Internet use «fairly» or «very often» (Livingstone, Haddon, Görzig, & Ólafsson, 2011). Furthermore, the results from EU NET ADB (a research project funded by the European Commission’s Safer Internet Programme) have recently been published. Based on a representative sample of 13,284 adolescents aged 14-17 from 7 European countries, this study reported that 1.2% of the total sample present Internet addictive behaviour (IAB), while 12.7% are said to be at risk (Tsitsika, Tzavela, Mavromati, & the EU NET ADB Consortium, 2012).
In Spain, several studies have reported different prevalence rates for young people: from 3.7% of problematic Internet use in adolescents aged 14-18 from Madrid (Estévez, Bayón, de la Cruz, & Fernández-Líria, 2009), to 9.9% of excessive users among university students (Muñoz-Rivas, Fernández, & Gámez-Guadix, 2010). Other researchers suggested intermediate values, such as the 5% of Internet overusers referred by Viñas et al. (2002), the 6.1% with frequent problems because of their Internet use found by Carbonell et al. (2012) or the 6.2% of pathological users found by Jenaro, Flores, Gómez-Vela, González-Gil, & Caballo (2007).
In spite of the considerable knowledge now available, it is diffi cult to integrate the different studies’ results and to properly diagnose the current situation. It is not currently possible to reach an agreement on which is the proper term to describe the problem. As stated in the study of Gámez-Guadix et al. (2013), this pattern has been called Compulsive Use, Problematic Internet Use, Internet Addiction, Maladaptive Internet Use or Pathological Internet Use. Although these terms are often used interchangeably, there is still signifi cant controversy, so, in our view, the term used in each case should be chosen with caution.
The infl uence of distinct sample sizes used, the different reference population, the dissimilar procedures of data collection, and the lack of agreed criteria about what should be considered addiction or not has resulted in a high level of confusion. The multiplicity of evaluation instruments used is also another issue that needs to be resolved (Beranuy, Chamarro, Graner, & Carbonell, 2009; Ceyhan, Ceyhan, & Gûrcan, 2007; De Gracia, Vigo, Fernández, & Marcó, 2002; Young, 1998a; Young, 1998b). Many of the scales developed so far have problems: (a) they are not all specifi c for adolescent population or their items do not correspond to teenager’s reality (Armstrong, Phillips, & Saling, 2000; Nichols & Nicki, 2004); (b) they do not provide data on their psychometric properties (Echeburúa et al., 2009); (c) the size of the samples used for empirical validation is small (Lam-Figueroa et al., 2011); (d) it would be diffi cult to use them as screening tools given their high number of items (Davis, Flett, & Besser, 2002; García del Castillo et al., 2008), or because they do not provide cut-off points, neither sensitivity and specifi city values (Gámez-Guadix et al., 2013; Meerkerk, Van Den Eijnden, Vermulst, & Garretsen, 2009); (e) there are no Spanish versions (Demetrovics, Szeredi, & Rózsa, 2008); and (f) the cultural environment where they were developed seems to have very little to do with Spanish culture (Huang, Wang, Qian, Zhong, & Tao, 2007; Lin & Tsai, 2002).
In short, the research that has been carried out to date with its accompanying abundant literature is contrasted by the lack of agreement on both the conceptualization and evaluation of problematic Internet use. Moreover, some researchers, such as Aboujaoude (2010), have suggested that many of the existing scales have not received adequate psychometric testing.
The present study therefore has a markedly applied objective being the development of an early detection tool of problematic Internet use in adolescents, understanding this as a maladaptive behaviour pattern with interferences in daily life. This instrument must be backed by suffi cient theoretical support (integrating the different scales and contributions collected in the literature), based on empirical evidence, and have acceptable psychometric properties, both in terms of reliability, validity, sensitivity and specifi city. Furthermore, the study aimed to generate a useful instrument on a practical level, with particular emphasis on conciseness, easy applicability and to contain items appropriate to the adolescent population, and to Spanish culture. Such a new instrument will constitute a benefi t in two ways: on the one hand, it will provide data to gauge the magnitude of the problem, reporting existing risk levels, and on the other hand, it will promote early detection or screening of potential risk cases.
Method
Participants
In pursuit of its purpose, a selective methodology and a cross design were used, carrying out a survey of the Compulsory Secondary Education students from Galicia. For the sample selection, two-stage sampling was used: by cluster sampling, for the selection of the fi rst-level units (secondary schools), and by quotas of Gender and Grade, for the selection of the second-level units (individuals). For data collection, a total of 29 secondary schools, public as well as private/subsidized, were randomly selected, in both urban and rural area and from the four Galician provinces, respecting population quotas. The fi nal sample consisted of 2,339 Compulsory Secondary Education students from Galicia, 1,171 girls and 1,168 boys, between the ages of 11 and 18 (M = 13.77, SD = 1.34). Of these, 1,619 attended public schools and 720 attended private or subsidized schools; 1,239 studied in lower secondary education (1st and 2nd grade) and 1,100 were in upper secondary education (3rd and 4th grade).
Instruments
Data collection was undertaken through the application of a questionnaire (developed for a broader study) which included a screening scale of problematic Internet use, comprising 9 Likert-type items, with fi ve answer options ranging from 0 “Strongly disagree” to 4 “Strongly agree”, extracted from the review of the literature (see Table 1). For the selection of the items, particular attention has been paid to their theoretical and empirical support and, at the same time, to the possibility of using them for screening problematic Internet use, taking into account some of the Internet Gaming Disorder criteria (symptoms of withdrawal, endangering their academic functioning…).
Procedure
study, and repeatedly assured of the complete anonymity and confi dentiality of their responses. This study was carried out with the consent and cooperation from both the school leadership and respective parents’ associations, and the participation was entirely voluntary.
Data analysis
Descriptive statistics (M, SD, skewness and kurtosis indexes) were calculated to study item distribution at a univariate level, and mul tivariate normality was tested via Mardia’s coeffi cient. A Confi rmatory Factor Analysis (CFA) was conducted to study the factorial structure of the scale, and maximum likelihood (ML) was the chosen method. Cronbach’s alphas were estimated to assess internal consistency. In addition, a specifi city and sensitivity analysis, and a Receiver Operating Characteristic (ROC) Curve analysis were included. All statistical analyses were carried out by using the IBM SPSS Statistics 20 and IBM SPSS Amos 20.
Results
Firstly, with regard to Internet use habits, it is worth emphasizing that 60.4% of teenagers affi rmed they accessed the Internet every day or almost every day. In addition, 26.8% connected to the Internet occasionally (once or twice a week). A large proportion of adolescents were usually connected between 1 and 2 hours a day (45.8%), although 10.5% acknowledged they spent more than 3 hours a day. The time slot 16.00-21.00 was the most usual time zone of Internet connection (56.8%), although 39.2% of teenagers were connected between 21.00-00.00 as well, and almost 6% from midnight. 36.2% of the interviewees stated that they are not on Internet all the time they would like. Another interesting fact is that 55% of adolescents had Internet access in their own room, and 15.8% used their mobile phone to navigate the Internet.
With regard to the role of parents, it should be noted that 54% of the interviewees said their parents do not control their Internet use at all. Furthermore, 77.9% of young Internet users had rarely or never had an argument with their parents due to their Internet use, whereas 6.3% declared having problems many times, and 14.6% occasionally.
Finally, in relation to some potentially dangerous uses, 64.3% of adolescents usually downloaded music, movies, photos… from the Internet, 40.2% usually uploaded photos to Facebook or Tuenti, and 20.4% admitted that they visit some age-inappropriate webpages.
With regard to the testing of the scale itself, Table 2 shows descriptive statistics for each of the 9 initial version items. The highest averages corresponded to item 6 (When I am online, I feel time fl ies, and I realise I have been on the Internet for hours), and item 1 (It is important for me to connect daily to Facebook, Tuenti…), scoring 2.39 and 2.27, respectively. Meanwhile, the lowest average corresponded to item 4 (At times, I prefer to stay connected to the Internet instead of being with my family or friends), scoring 0.95. Related to the variability of response, item 9 (Sometimes I feel more comfortable chatting on Facebook, Tuenti… than talking face to face with people) had the most heterogeneous answers (SD = 1.47), although there were no large differences across items.
In connection with frequency distribution, items 2, 4, 5, 7, 8 and 9 had positive standardized skewness values, while items 1 and 6 had a strong negative skewness, with absolute values greater than 3. Regarding kurtosis, many of the items showed a platykurtic distribution. Mardia’s coeffi cient was 22.07, indicating no multivariate normality.
Table 1
Items of the initial screening scale and sources
Items Sources
1. Para mí es importante poder conectarme diariamente a Facebook, Tuenti… [It is important for me to connect daily to Facebook, Tuenti…]
Davis et al., 2002
2. En algunas ocasiones he perdido horas de sueño por usar Internet [Sometimes I have lost hours of sleep due to Internet use]
Armstrong et al., 2000
Chen, Weng, Su, Wu, & Yang, 2003 Demetrovics et al., 2008 García del Castillo et al., 2008 Huang et al., 2007 Young, 1998a
3. A veces me conecto más de lo que debiera [Sometimes I get online more than I should]
Davis et al., 2002 Echeburúa et al., 2009
4. En ocasiones prefi ero quedarme conectado/a a Internet en lugar de estar con mi familia o amigos/as [At times, I prefer to stay connected to the Internet instead of being with my family or friends]
Chen et al., 2003
García del Castillo et al., 2008 Young, 1998b
5. En ocasiones me pongo de mal humor por no poder conectarme [At times, I get in a bad mood because of not being able to connect to the Internet]
Demetrovics et al., 2008 García del Castillo et al., 2008 Lam-Figueroa et al., 2011 Young, 1998b
6. Cuando estoy conectado/a siento que el tiempo vuela y cuando me doy cuenta llevo horas en Internet [When I am online, I feel time fl ies, and when I realise I have been on the Internet for hours]
Beranuy et al., 2009 Huang et al., 2007
7. He descuidado mis tareas escolares por conectarme a Internet [I have neglected my homework due to Internet use]
Beranuy et al., 2009 Chow, Leung, Ng, & Yu, 2009 De Gracia et al., 2002 García del Castillo et al., 2008 Huang et al., 2007 Young, 1998a
8. He dejado de hacer cosas importantes para estar conectado/a [I have stopped doing important things to get connected]
Beranuy et al., 2009 De Gracia et al., 2002 Echeburúa et al., 2009
9. A veces me siento más cómodo chateando por Facebook, Tuenti… que hablando cara a cara con la gente [Sometimes I feel more comfortable chatting on Facebook, Tuenti… than talking face to face with people]
Beranuy et al., 2009 Caplan, 2002
García del Castillo et al., 2008
Table 2
Descriptive statistics of the initial screening scale
M SD Skewness Kurtosis CHI
Item 1 2.27 1.38 0-4.12 -10.36 .49
Item 2 1.10 1.45 -17.57 0-5.36 .59
Item 3 2.02 1.46 0-0.12 -12.53 .53
Item 4 0.95 1.23 -21.77 -0 3.32 .53
Item 5 1.42 1.43 -10.71 0-9.29 .61
Item 6 2.39 1.42 0-5.66 -11.22 .49
Item 7 1.03 1.29 -18.98 0-1.11 .58
Item 8 1.05 1.28 -19.21 0-0.19 .63
In order to analyze the relationship of each item to the whole scale, Corrected Homogeneity Index (CHI) was calculated, obtaining values between .49 and .65 for all items, except item 9 for which the CHI was .38. The internal consistency of this initial version was calculated by Cronbach’s coeffi cient alpha. The total value obtained was very acceptable (.84).
The responses were subjected to factor analysis to examine the psychometric properties of the screening scale of problematic Internet use. A CFA was conducted to confi rm one-dimensional factorial structure proposed by Young (1998b). Parameters were estimated using ML, given that studies such as those performed by Curran, West, & Finch (1996) and Tomás & Oliver (1998) have shown that maximum likelihood is robust against violations of normality when samples are large, and in any case, a worse data fi t than real one would be provided. Every estimated parameter (see Figure 1) was statistically signifi cant (p<.01), and factor loadings (λ) showed high values, except in the case of item 9.
Goodness-of-fi t was assessed using different indicators, as recommended by researchers such as Byrne (2009) or Kline (2005): χ2, χ2/degrees of freedom, Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Comparative Fit Index (CFI), Normed Fit Index (NFI), Tucker Lewis Index (TLI) and Root Mean Square Error of Approximation (RMSEA). The results from the different fi t indices used seemed to show that the scale fi ts moderately well to the theoretical model (see Table 3). The values of GFI, AGFI, CFI and NFI were higher than .90, the TLI was .89, and the RMSEA (.089) was higher than the reference value recommended by Hu & Bentler (1999).
Because of its low CHI (.38) and its low factor loading in CFA (.41), item 9 was removed. Furthermore, a detailed analysis of modifi cation indices suggested to take into account two specifi c parameters related to the measurement error correlations between
item 3 and 6 (δ3-δ6), and item 7 and 8 (δ7-δ8). In order to test the stability of the respecifi ed model, a cross-validation attempt was carried out. The original database was divided randomly into two different subsamples and the same statistical procedure was applied to compare both subgroups. Empirically corroborated goodness-of-fi t indices for the model were high and very similar in both halves (see Table 3). The values of GFI, AGFI, CFI, NFI and TLI were higher than .95, and the RMSEA didn’t exceed .08. Furthermore, estimated parameters were similar in both random subsamples (Figure 2 and 3) and measurement error correlations (δ3-δ6 and δ7-δ8) were statistically signifi cant as well. Finally, the internal consistency of the scale was reanalyzed, obtaining a very acceptable index (α = .83).
The goodness-of-fi t of the model have been substantially improved (see Table 3). With respect to the descriptive statistics of the total sample, taking into account that the 8-item fi nal scale has a theoretical minimum score of 0 and a maximum of 32 points, the mean was 12.22 and the standard deviation was 7.44.
Finally, as there are no consensus diagnostic criteria yet to identify a clinical sample, and in order to try to explore its screening
l1
l2
l3
l4
l5
l6
l7
l8
l9 e1
e2
e3
e4
e5
e6
e7
e8
e9
SCALE ,51
,67
,58
,60
,66
,53
,69
,74
,41
Figure 1. Confi rmatory factor analysis: Initial Structural Model
Table 3
Confi rmatory factor analysis, model fi t indices
χ2 gl p χ2/gl GFI AGFI CFI NFI TLI RMSEA [CI]*
Initial model 451.16 27 <.001 16.71 .94 .91 .91 .91 .89 .089 [.082-.096]
Respecifi ed model (Half 1) 125.66 18 <.001 06.98 .97 .94 .95 .94 .92 .077 [.065-.090]
Respecifi ed model (Half 2) 106.44 18 <.001 05.91 .97 .95 .96 .96 .94 .070 [.058-.083]
* 90% Confi dence interval for RMSEA
e1 l1
e2 l2
e3 l3
e4 l4
e5 l5
e6 l6
e7 l7
e8 l8
SCALE ,53
,63
,52
,60
,68
,53
,61
,67 ,18
,33
Figure 2. Confi rmatory factor analysis: Final Structural Model (Half 1)
e1 l1
e2 l2
e3 l3
e4 l4
e5 l5
e6 l6
e7 l7
e8 l8
SCALE ,53
,72
,60
,60
,68
,50
,67
,72 ,20
,25
capacity, the available sample was divided into two groups: (a) a fi rst one that could be considered “Normal group” and (b) a second one that could be considered “Risk group”, consisting of the individuals who connect to the Internet every day, usually more than fi ve hours per day, and have frequent arguments with their parents for this reason. The sensitivity and specifi city obtained for different cut-off points are shown in Figure 4. As can be noted, values 18 and 19 achieved the best balance between both indicators. For a score of 19, a sensitivity of 80% and a specifi city of 81.2% were obtained, which means the scale would be able to detect 80% of true positives and to reject 81.2% of true negatives. In addition a Receiver Operating Characteristic (ROC) curve analysis was performed to determine optimal cut-off value, obtaining an area under the curve of .90 (see Figure 5). Taking as a cut-off point a score of 19, and applying the screening scale to the available sample, this would mean that 19.9% of Galician adolescents would be classifi ed as potentially problematic Internet users.
Discussion
The current study was designed to develop a screening or early detection tool of problematic Internet use in adolescents. Based on analyses carried out from a sample of 2,339 students from Galicia, a short scale (8 items) developed from the main previous tests (Beranuy et al., 2009; Davis et al., 2002; Echeburúa et al., 2009; García del Castillo et al., 2008; Young, 1998a; Young, 1998b) is presented as a screening instrument with acceptable psychometric properties, in terms of reliability, validity, sensitivity and specifi city. This simple and easy to apply tool is adapted to the Spanish cultural context and to the youth language, which means it has a great practical potential. As a screening tool, its use enables the gradation of adolescents on a risk continuum, but it is not diagnostic due to the lack of agreed established criteria and the necessary clinical assessment.
Its application on a representative sample of Compulsory Secondary School students from Galicia revealed that 19.9% of adolescents would be problematic Internet users. Similar prevalence estimates of Maladaptive Internet Use in adolescents (MIU = 13.5%) and Pathological Internet Use (PIU = 4.4%) were recently reported by Durkee et al. (2012). Therefore, our results provide the concerning suggestion that at least 15,000 Galician adolescents are “at risk”, strongly reinforcing the need to implement preventive measures.
We are aware of the limitations of this study and we therefore emphasise that it should be considered as an empirical approach the primary purpose of which was to contribute to improved knowledge of the problem and to its evaluation. It is impossible to access clinical samples to test the scale properties because of lack of clinical consensus on diagnostic criteria and no offi cial recognition yet of Internet addiction. However, the fact remains that the scientifi c community, institutions, and society as a whole are demanding a proactive approach and attitude towards a situation that could become a real problem. This work should be considered a starting point to further address adolescent Internet use in a serious and rigorous way. Future research will lead to an improvement of the developed screening scale and to its clinical validation. In this regard, one of the greatest challenges in this research fi eld is to achieve consensus on what to call (Compulsive Use, Problematic Internet Use, Internet Addiction…) and how conceptualize this pattern, identifying the criteria to use in evaluating. Otherwise the comparison among different studies will remain impossible. 100
90 80 70 60 50 40 30 20 10 0
14 15 16 17 18 19
94.3 94.3 91.4 91.4 85.7
80
60.5 65
70 74.1 78
81.2
Sensitivity Specificity
Figure 4. Sensitivity and Specifi city values for different cut-off points
1,0
0,8
0,6
0,4
0,2
0,0
0,0 0,2 0,4 0,6 0,8 1,0
1. Specificity
Sensitivity
Figure 5. ROC curve analysis for the fi nal screening scale
References
Aboujaoude, E. (2010). Problematic internet use: An overview. World Psychiatry, 9, 85-90.
American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental Disorders. Fifth edition. Washington, DC: Author.
Armstrong, L., Phillips, J.G., & Saling, L.L. (2000). Potential determinants
Beranuy, M., Chamarro, A., Graner, C., & Carbonell, X. (2009).Validación de dos escalas breves para evaluar la adicción a Internet y el abuso de móvil [Validation of two brief scales for Internet addiction and mobile phone problem use]. Psicothema, 21, 480-485.
Byrne, B.M. (2009). Structural equation modeling with AMOS: Basic concepts, applications, and programming (2nd ed.). London: Psychology Press.
Cao, F., & Su, L. (2006). Internet addiction among Chinese adolescents: prevalence and psychological features. Child: Care, Health and Development, 33, 275-281.
Caplan, S.E. (2002). Problematic Internet use and psychosocial well-being: Development of a theory-based cognitive-behavioral measurement instrument. Computers in Human Behavior, 18, 553-575.
Carbonell, X., Chamarro, A., Griffi ths, M., Oberst, U., Cladellas, R., & Talarn, A. (2012). Problematic Internet and cell phone use in Spanish teenagers and young students. Anales de Psicología, 28, 789-796. Castellana, M., Sánchez-Carbonell, X., Graner, C., & Beranuy, M.
(2007). El adolescente ante las tecnologías de la información y la comunicación: Internet, móvil y videojuegos [Adolescents and information and communications Technologies: Internet, mobile phone and videogames]. Papeles del Psicólogo, 28, 196-204.
Ceyhan, E., Ceyhan, A.A., & Gûrcan, A. (2007). The validity and reliability of the Problematic Internet Usage Scale. Educational Science: Theory and Practice, 7, 411-416.
Chen, S.H., Weng, L.C., Su, Y.J., Wu, H.M., & Yang, P.F. (2003). Development of Chinese Internet Addiction Scale and its psychometric study. Chinese Journal of Psychology, 45, 279-294.
Chow, S.L., Leung, G.M., Ng, C., & Yu, E. (2009). A Screen for Identifying Maladaptive Internet Use. International Journal of Mental Health & Addiction, 7, 324-332.
Curran, P.J., West, S.G., & Finch, J.F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16-29.
Davis, R.A. (2001). A cognitive-behavioral model of pathological Internet use. Computers in Human Behavior, 17, 187-195.
Davis, R.A., Flett, G.L., & Besser, A. (2002). Validation of a new scale for measuring problematic Internet use: Implications for pre-employment screening. Cyberpsychology & Behavior, 5, 331-345.
De Gracia, M., Vigo, M., Fernández, M.J., & Marcó, M. (2002). Problemas conductuales relacionadas con el uso de Internet: un estudio exploratorio [Behavioral problems related with Internet usage: An exploratory study]. Anales de Psicología, 18, 273-292.
Demetrovics, Z., Szeredi, B., & Rózsa, S. (2008). The three-factor model of Internet addiction: The development of the Problematic Internet Use Questionnaire. Behavior Research Methods, 40, 563-574.
Durkee, T., Kaess, M., Carli, V., Parzer, P., Wasserman, C., Floderus, B., et al. (2012). Prevalence of pathological internet use among adolescents in Europe: Demographic and social factors. Addiction, 107, 2210-2222. Echeburúa, E., Labrador, F.J., & Becoña, E. (2009). Adicción a las nuevas
tecnologías en adolescentes y jóvenes [Addiction to new technologies in adolescents and young adults]. Madrid: Ediciones Pirámide. Estallo, J.A. (2001). Usos y abusos de Internet [The uses and abuses of
Internet]. Anuario de Psicología, 32(2), 95-108.
Estévez, L., Bayón, C., de la Cruz, J., & Fernández-Líria, A. (2009). Uso y abuso de Internet en adolescentes [Use and abuse of Internet in adolescents]. In E. Echeburúa, F.J. Labrador & E. Becoña (Eds.), Adicción a las nuevas tecnologías en adolescentes y jóvenes (pp. 101-130). Madrid: Ediciones Pirámide.
Gámez-Guadix, M., Orue, I., & Calvete, E. (2013). Evaluation of the cognitive-behavioral model of generalized and problematic Internet use in Spanish adolescents. Psicothema, 25, 299-306.
García del Castillo, J.A., Terol, M.C., Nieto, M., Lledó, A., Sánchez, S., Martín-Aragón, M., et al. (2008). Uso y abuso de Internet en jóvenes universitarios [Use and abuse of the Internet in university students]. Adicciones, 20, 131-142.
Holtz, P., & Appel, M. (2011). Internet use and video gaming predict problem behavior in early adolescence. Journal of Adolescence, 34, 49-58. Hu, L., & Bentler, P.M. (1999). Cutoff criteria for fi t indexes in covariance
structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55.
Huang, Z., Wang, M., Qian, M., Zhong, J., & Tao, R. (2007). Chinese Internet addiction inventory: Developing a measure of problematic Internet use for Chinese college students. Cyberpsychology & Behavior, 10, 805-811.
Jenaro, C., Flores, N., Gómez-Vela, M., González-Gil, F., & Caballo, C. (2007). Problematic Internet and cellphone use: Psychological, behavioral, and health correlates. Addiction Research & Theory, 15, 309-320.
Kaltiala- Heino, R., Lintonen, T., & Rimpelä, A. (2004). Internet addiction? Potentially problematic use of the Internet in a population of 12-18 years-old adolescents. Addiction Research & Theory, 12, 89-96. Kline, R.B. (2005). Principles and Practice of Structural Equation
Modeling (2nd ed.). New York: The Guilford Press.
Kormas, G., Critselis, E., Janikian, M., Kafetzis, D., & Tsitsika, A. (2011). Risk factors and psychosocial characteristics of potential problematic and problematic internet use among adolescents: A cross-sectional study. BMC Public Health, 11, 595-602.
Lam-Figueroa, N., Contreras-Pulache, H., Mori-Quispe, E., Nizama-Valladolid, M., Gutiérrez, C., Hinostroza-Camposano, W., et al. (2011). Adicción a Internet: desarrollo y validación de un instrumento en escolares adolescentes de Lima, Perú [Internet addiction: Development and validation of an instrument in adolescent scholars in Lima, Peru]. Revista Peruana de Medicina Experimental y Salud Pública, 28, 462-469.
Lin, S.S.J., & Tsai, C.C. (2002). Sensation seeking and internet dependence of Taiwanese high school adolescents. Computers in Human Behavior, 18, 411-426.
Livingstone, S., Haddon, L., Görzig, A., & Ólafsson, K. (2011). Risks and safety on the internet: The perspective of European children. Full
fi ndings. London: LSE, EU Kids Online.
Meerkerk, G-J., Van Den Eijnden, R.J.J.M., Vermulst, A.A., & Garretsen, H.F.L. (2009). The Compulsive Internet Use Scale (CIUS): Some psychometric properties. Cyberpsychology & Behavior, 12, 1-6. Muñoz-Rivas, M.J., Fernández, L., & Gámez-Guadix, M. (2010). Analysis
of the indicators of pathological Internet use in Spanish university students. The Spanish Journal of Psychology, 13, 697-707.
Nichols, L.A., & Nicki, R. (2004). Development of a psychometrically sound internet addiction scale: A preliminary step. Psychology of Addictive Behaviors, 18, 381-384.
Sánchez-Martínez, M., & Otero, Á. (2010). Usos de Internet y factores asociados en adolescentes de la Comunidad de Madrid [Internet and associated factors in adolescents in the Community of Madrid]. Atención Primaria, 42, 79-85.
Tomás, J.M., & Oliver, A. (1998). Efectos de formato de respuesta y método de estimación en el análisis factorial confi rmatorio [Response format and method of estimation effects on confi rmatory factor analysis]. Psicothema, 10, 197-208.
Tonioni, F., D’Alessandris, L., Lai, C., Martinelli, D.,Corvino, S., Vasale, M., et al. (2012). Internet addiction: Hours spent online, behaviors and psychological symptoms. General Hospital Psychiatry, 34, 80-87. Tsitsika, A., Tzavela, E., Mavromati, F., & the EU NET ADB Consortium.
(Ed.) (2012). Research on Internet Addicitive Behaviours among European Adolescents (EU NET ADB Proyect). Athens: National and Kapodestrian University of Athens.
Viñas, F. (2009). Uso autoinformado de Internet en adolescentes: perfi l psicológico de un uso elevado de la red [Self-reported use of Internet among adolescents: Psychological profi le of elevated internet use]. International Journal of Psychology and Psychological Therapy, 9, 109-122.
Viñas, F., Juan, J., Villar, E., Caparros, B., Pérez, I., & Cornella, M. (2002). Internet y psicopatología: las nuevas formas de comunicación y su relación con diferentes índices de psicopatología [Internet and psychopathology: New forms of communication and their connection to several psychopathological signs]. Clínica y Salud, 13, 235-256. Young, K.S. (1998a). Caught in the Net: How to recognize the signs of
Internet addiction and a winning strategy for recovery. New York: John Wiley & Sons.
