9.3.1
Goodman’s Strengths and Difficulties Questionnaire
The main outcome measure used in the analysis is Goodman’s Strengths and Difficulties Questionnaire (SDQ) (Goodman, 1997). The SDQ is a brief behavioural screening questionnaire for children aged from 2 to 16.The SDQ contains five separate scales and asks about 25 attributes of social, emotional and behavioural development. The scales are Emotional Symptoms, Conduct Problems, Hyperactivity and inattention, Peer Relationship Problems, and Pro-social Behaviours. The five scales each contain five statements, to which the respondent answers as to whether the statement is ‘not true’,
‘somewhat true’ or ‘certainly true’ of the child. The first four (negatively-rated) scales can be added together to create a ‘total difficulties’ scale. The Pro-social Behaviours scale is a positively-rated scale, whereby a higher score equates to a lower level of difficulties. Cut-off scores are provided which allocate children into one of three groups: no difficulties, possible difficulties or likely difficulties. In a normal population, the no difficulties group would contain 80% of children, with the remaining two groups each containing 10%. For the present analysis two of these categories will be collapsed creating two groups – those with likely
94 difficulties (also known as an abnormal score) and all others (the no difficulties or normal group combined with children who scored in the possible
difficulties/borderline group). This decision was taken due to the relatively poor predictive value of the ‘possible difficulties’ group (Goodman, 1997; Goodman et al., 2000).
The SDQ has been found to have good predictive validity. Children who scored in the likely difficulties range of the SDQ (as rated by parents or teachers) had increased odds of 15 for being subsequently diagnosed with a psychiatric disorder 4-6 months later. Children who rated themselves as having likely difficulties on the self-complete version of the SDQ, had odds of a psychiatric diagnosis 6 times higher (Goodman, 2001). In a normal population sample, the SDQ produces high numbers of true negatives (specificities and negative predictive values at c.95%) i.e. the proportion of children screened who don’t have problems on the SDQ and really don’t have problems (i.e. meeting the clinical diagnosis criteria) in real life, but it also produces lower proportions of true positives (the proportion of children with a likely difficulty on the SDQ who also have a diagnosis in real life), at just 35%. As Goodman points out though, screening tests often accept this level of risk in terms of the identification of false positives, as the priority is to reduce the rate of false negatives (Goodman, 2001). Similar results have been found in a number of other contexts (Wichstrom et al., 2012; Bourdon et al., 2005; Fleitlich-Bilyk & Goodman, 2004).
The SDQ is available in three separate versions for completion by parents, teachers and in a self-completion version for older children (aged 11-16). There is a separate version for 3-4 year old children, which replaces the items ‘often lies or cheats’ with ‘often argumentative with adults’ and ‘Steals from home, school or elsewhere’ with ‘can be spiteful to others’. For the present study the 4-16 year old version of the questionnaire was used in 2010, the first year of data collection. In later years this was changed to the 3-4 year old version following staff feedback regarding some of the items in the older version being age inappropriate. In particular, there was a view from staff that although some children may put toys in their pockets and go home with them (essentially ‘stealing’), the intent was not there, and so this question was not felt to be developmentally appropriate (White, Connelly, Thompson, & Wilson, 2013). The result of this is that scores on the Conduct Problems domain may be lower than
95 expected in 2010, as staff were more reluctant to rate children as having
difficulties in these areas.
Data collection at pre-school and P3 used the teacher-rated version of the questionnaire. At pre-school data collection at age 4-5, the SDQ was completed by a pre-school staff member (generally the Child Development Officer) who has known the child for at least six months. This is sometimes completed in
collaboration with other nursery staff. At P3 (age 7-8), the SDQ was completed by the class teacher.
9.3.2
Potential Issues with the SDQ
The SDQ was designed as a shortened behavioural screening version, based on Rutter’s longer questionnaire (Goodman, 1997). Whilst the SDQ has been
validated in different settings (Muris, Meesters, & van den Berg, 2003; Goodman, 1997; Hawes & Dadds, 2004), there remains some criticism of use of the scale from some quarters. The scale is substantially shorter than many of the
commonly used scales, for example the Child Behaviour Checklist (CBCL) (Achenbach & Edelbrock, 1991), though Goodman’s own work on comparing validity between the SDQ and CBCL concludes that the SDQ is at least as good at detecting problems, and detects more on the inattention and hyperactivity scale (Goodman & Scott, 1999). These findings are supported by evidence comparing the German versions of the SDQ and CBCL (Klasen et al., 2000).
As discussed above, the SDQ is being completed by different raters at different times. The SDQ has the highest validity when completed by all three types of respondents simultaneously. While it would be highly desirable to have multiple informants at each time point in the Glasgow study, the resources available do not allow for this. There has been a considerable amount of research about the extent to which teacher, parent and self-report versions of the SDQ overlap (Collishaw, Goodman, Ford, Rabe Hesketh, & Pickles, 2009; Goodman, Ford, Corbin, & Meltzer, 2004). Goodman found a substantial overlap in the three informants’ scores, ranging from r=0.35 (for teacher-child correlations in 1999) to r=0.50 (for parent-teacher and parent-child correlations in 2004). Agreement in SDQ scores between parents and teacher was substantially higher for boys than for girls (r=0.52 for boys in 2004, compared with r=0.45 for girls in 2004) (Collishaw et al., 2009). The concurrent validity study which ran alongside the
96 SDQ Pre-school pilot in Glasgow investigated the overlap between teacher and parent ratings of the same child. SDQs were given out to parents in a sub-sample of 24 nurseries in Glasgow (n=676). Forty percent of carers (n=273) completed the SDQ. It was possible to match carer and teacher SDQ forms in 60% of cases (n.=180), of which 174 had a complete set of information. There was broad agreement between parents and teachers as to whether the child was
experiencing ‘probable difficulties’ on each scale, with agreement greatest on the Emotional Symptoms, Hyperactivity / inattention and Total Difficulties scales (91-92%). Teachers were less likely to find problems on the Conduct and Peer relations scales, compared with parents. The study is limited in its
conclusions however, due to the small numbers of matched results. Goodman’s work suggests that, while the SDQ prediction works best when completed by both carers and teachers, where only one adult completes the SDQ, the parent and teacher versions provide roughly equal predictive value (Goodman et al., 2004).
It is also the case that different informants witness children in different settings, where real differences may be observed, for example, a teacher may witness far more social interaction between the child and other children and therefore pick up more problems in the peer problems or pro-social domains. In the Glasgow pre-school concurrent validity study however, teachers were less likely to pick up peer problems, though again, numbers here were small. Furthermore, in Primary school, children are expected to sit and concentrate on tasks for longer, so teachers may be more likely to observe hyperactivity or inattention problems, compared with parents or even nursery staff. It has also been suggested in the literature that teachers may have more of an idea of what ‘normal’ development looks like, and may therefore be more likely to spot children who appear to be not developing normally (Stone, Otten, Engels, Vermulst, & Janssens, 2010). On the other hand, there has been some concern raised by nursery staff in the Glasgow qualitative feasibility study about the labelling of children (White et al., 2013). It is feared that this may lead some nursery staff to under-report some problems. There is also a concern that partnership provider nurseries may be under pressure, due to the way they are funded, not to report too many problems. Comparisons with national and international norms should help to shed light on this issue.
97 A further consideration in this type of research is the impact of social
desirability. Social desirability is said to “reflect[s] the tendency on behalf of
the subjects to deny socially undesirable traits and to claim socially desirable ones, and the tendency to say things which place the speaker in a favourable light” pg.264 (Nederhof, 1985). It may be that some staff members would like to
portray the behaviour of children in their establishment more positively than may be the reality, particularly as this information is collected by the Education Services department, through which pre-school establishments receive their partnership funding and schools are monitored. In contrast, it could be that nursery staff and teachers rate their children more negatively, in the hope that this will attract additional resources to the school or nursery in which they work. One would normally compare data to national or international norms, in order to get a gauge of any bias, however, there is a lack of available data broken down by age and a potential ‘Glasgow Effect’ may mean that Glasgow data are
actually different rather than reflecting bias in the data.