Autism spectrum disorders (ASDs) are neurodevelopmental, lifelong conditions diagnosed using a set of behavioural criteria.1ASD is known to affect at least 1% of the child and adult population.2–4There is wide
variation in the progress made by individuals with ASD, so that many individuals have significant lifelong needs for support. The burden and cost to the individual, family and broader society are very high, with the economic costs in the UK estimated to be £28B per year.5
In light of increased awareness about the prevalence of ASD, and the emphasis on early identification and diagnosis, it is important that health, education and social-care services provide evidence-based interventions and early support for individuals with ASD, and their families, carers and teachers. In the past decade there has been an increase in ASD intervention research, with recent improvement in the quality of studies.6,7The ASD early intervention literature is largely focused on promotion of social communication
skills, with less emphasis on interventions for restricted and repetitive behaviours (RRBs). It also includes interventions focused on the high rates of co-occurring behaviours and problems (e.g. sleep, faddiness about food, aggression to others, toileting difficulties).8,9One problem for the interpretation of research
findings is the multitude of different measurement tools that have been used in collecting evidence of progress and outcomes. Furthermore, longitudinal studies highlight the variation in individual
developmental pathways.10–12The changes in prevalence are due, in part, to earlier recognition of ASD in
children in the average range of ability, with likely effects on the pattern of outcomes.13The literature thus
presents a large set of measures, inconsistently used, of varying relevance and with variable or indeed no evidence of their psychometric properties.
What should be measured?
There are several ways to consider the question of what to measure, including what government departments need in order to measure progress and outcomes, what matters to parents and individuals with ASD, and the theoretical basis of ASD, which has implications regarding important domains to measure.
The UK Chief Medical Officer’s 2012 report focused on Child Health,14and discussed the poor educational,
health and employment outcomes for children with neurodisability. In recent years, there has been consultation about the UK National Health Service Outcomes Framework 2011/12,15part of a strategy
that aims to deliver‘the outcomes that matter most to people’, using patient-reported outcome measures. The Kennedy report‘Getting It Right for Children and Young People’16highlighted the need to identify a
common vision between families and professionals for what services are seeking to achieve for children. Measuring outcomes that are valued by families is central to that vision, which, in turn, will influence what services are provided and how, and potentially what services and interventions are prioritised for research evaluation. A recent National Institute for Health Research (NIHR) study has reported agreement on what are the valued outcomes of care for children with neurodisability,17but it is not clear whether or not these
The aims of our MeASURe (Measurement in Autism Spectrum disorder Under Review) review are to identify the validity of tools and outcome measures used in measuring and monitoring young children with ASD, and to consider how well these reflect and measure issues of importance for patients and carers (seeAppendix 1). To achieve this we have:
l identified the tools reported in literature on quantitative research involving children of up to approximately 6 years of age with ASD (seeChapter 3)
l conducted a detailed systematic review of the measurement properties of tools within the major domains of development and functioning (seeChapter 4)
l synthesised evidence regarding the most robust and useful tools in these different domains (seeChapter 5)
l identified gaps in measurement of outcomes and made research recommendations.
An important part of the strategy has been to identify what people on the autism spectrum, and parents of children with ASD, think should be measured. As these stakeholders were involved at various stages throughout the project, there is no single section on‘patient and public involvement’in the report. Instead, parents and people on the autism spectrum have contributed particularly toChapters 2,5and6. InChapter 2, we address the issue of what outcomes should be measured.
Chapter 2
Development of the conceptual
framework
Introduction
Within the MeASURe project, we carried out a series of consultations with stakeholders, including professionals, parents of children with ASD and people on the autism spectrum, and a scoping review of qualitative literature. The aim was to identify (1) what outcomes should be measured when monitoring the progress of young children with ASD and (2) whether there is agreement between parents and professionals about the relative importance of what to measure. The review of how to measure those outcomes in order to progress towards an agreed battery of tools is presented in later chapters. The chapter is structured to incorporate:
1. general considerations for developing a conceptual framework in ASD for the review 2. findings from scoping relevant qualitative research with families
3. consultation with people who are on the autism spectrum 4. survey consultation with professionals
5. consultation with parents
6. consultation with multiple stakeholders at a Discussion Day. Valued outcomes
There exist recommended procedures for agreeing what should be a core set of outcomes in various fields of health care. As Williamsonet al.18note,
‘insufficient attention has been paid to the outcomes measured in clinical trials’. Consistency and interpretation will be improved if researchers always collect and report on core outcomes. The Core Outcome Measures in Effectiveness Trials (COMET) initiative funded by the Medical Research Council Network of Hubs for Trials Methodology Research aims to develop a set of evidence-based procedures for developing a core outcome set. The suggested steps involve:
l Step 1 Agree the scope of the area of health care. l Step 2 Identify existing knowledge about outcomes.
l Step 3 Involve key stakeholders, including patients and health-care providers.
l Step 4 Develop consensus about what to measure. Techniques for doing this in an inclusive and objective way are outlined in Williamsonet al.,18including how to determine when consensus has
been achieved.
A systematic review of studies that aimed to determine which outcomes to measure in clinical trials in children concluded that in most specialties no research had been undertaken.19
The scope for this review was determined in the Health Technology Assessment (HTA) commissioning brief (i.e. COMET step 1) and includes a potentially broader use of outcomes than only in trials. This chapter presents the work undertaken on steps 2 and 3, i.e. to identify priorities for child outcomes as valued by parents and professionals, and as explored in qualitative literature. Because of the complexity of ASD as a disorder, and the developmental context of measuring outcomes up to the age of 6 years, the MeASURe review adopted a further step of placing the findings of the consultation stages in a conceptual framework to guide the full review of tools for measurement. This framework also guided further consultation with stakeholders about the relative importance of outcomes to measure. MeASURe did not undertake a further formal process to develop consensus (step 4 above). It may be that‘consensus’would be hard to achieve but it would require further procedures. In principle, the choice of outcomes to focus upon depends on
Considerations for developing a conceptual framework in autism spectrum disorder
One important potential basis for a conceptual framework for valued outcomes for children with ASD is the International Classification of Functioning, Disability and Health for Children and Youth (ICF-CY),20so
that what is measured can be‘mapped’against domains of functioning (e.g. Learning, Communication, Self-Care) and participation (e.g. Relationships, Community Life).
The conceptual framework should also be influenced by an understanding of ASD. The behavioural characteristics of ASD are underpinned by genetic, brain structure and neuropsychological differences from typical development.21The conclusions of many studies have led to the revision of theDiagnostic and
Statistical Manual of Mental Disorders-Fifth Edition (DSM-5) published in May 2013, such that the impairments in ASD are best considered within two groupings: social/communication difficulties and restricted/repetitive interests and behaviours, including hypo- or hyper-responsiveness to sensory stimuli. The aetiological underpinnings for each of these broad domains of impairments may be different, and both may be targets for interventions targeting‘core’features of autism.
Another aspect of complexity in the understanding of ASD is that its measurement is affected by developmental considerations, i.e. children’s profile of skills and difficulties may look very different over time, and those trajectories will also be affected by levels of ability. As an obvious example, we cannot measure children’s social ability to make and keep friendships with validity until they are of an age at which that might be expected in typical development. However, there is some recent agreement on the core early impairments that may be observed. By the age of 2 years, differences in the development of children with ASD (from typically developing children and those with developmental delay) are evident in behaviours such as fearfulness, frustration and lack of co-operation, quick mood changes, and fine and gross motor skills.22This knowledge has been enhanced by recent studies of the early development of
baby siblings of children with autism (who have an increased chance of themselves developing ASD); for example, Zwaigenbaumet al.23,24reported unusual eye contact, a lack of visual attention, orientating to
name, imitation, social interest and emotional affect, and heightened sensory-orientated behaviours. These combinations of deficits have consequences for development of relationships, early language and play, and, in turn, for the content and targets of early intervention.
The complexity of understanding ASD is made even greater when considering the interaction between domains of development, and how deficits in one may impact upon another; for example, visual sensory overload may lead to avoidance, which reduces opportunities for visual learning and social experience, leading to poor social skills. Furthermore, there is evidence that pragmatic skills (such as social timing in interaction) are closely associated with particular types of behavioural profiles.25,26Thus the conceptual
framework for a review of outcome tools should consider both measurement of individual areas of functioning, which are likely to change over time, and also tools that bring together these separate areas into a more holistic assessment. It is particularly important to be able to create profiles for children with ASD, who often have difficulties in generalising learning between areas of skill and also generalising skills from one setting to another.
It is also important to detail other associated difficulties that are not unique to ASD but which, nevertheless, can play a major part in children’s development and the burden of care for families. These commonly include feeding and eating difficulties (resistance to certain food textures, faddiness about types and colours of food, etc.), behaviour and sleeping problems. Children who lack adequate nutrition and sleep are likely to be bad tempered and even more rigid in their thinking and behaviour. Furthermore, adaptive functioning may be more impaired in children with ASD than would be expected from their level of ability. Finally, as young children’s development is intimately affected by their environment, including the health, skills and resilience of parents and carers, it is important to include consideration of the impact on the family.
Although the conceptual framework developed over the process of consultation, and was informed by the separate activities described below, it is presented first for brevity and clarity (Table 1). The framework
TABLE 1 Conceptual framework for the MeASURe review
Domains Subdomains Constructs of interest
Body Functions and Structures/Impairments
Symptom severity Change in diagnostic category; autism severity; diagnostic scores used as measures of outcome
Social awareness Joint attention skills; imitation; social attention Restricted, repetitive
behaviour
Repetitive, stereotyped movements; repetitive use of objects; repetitive use of language; attention to detail; insistence on sameness
Sensory processing Hypersensitivity; hyposensitivity
Language Expressive language; receptive language; gestures
Cognitive ability IQ/developmental quotient; non-verbal ability; verbal ability/ reasoning
Attention Distractibility; impulsivity; hyperactivity Emotional regulation Happiness; irritability; distress; anxiety
Physical skills Poor co-ordination/gross motor skills; fine motor skills
Physical indicators Tics; gut/bowel symptoms; nutritional status; height and weight (growth); effectiveness of medication; adverse effects of medication; vaccination rates
Activity-Level Indicators Social communication Frequency/quality of initiations; pragmatics
Social functioning Attachment; interaction skills with other children; awareness of others’emotions
Play Levels of play (exploratory to symbolic); organises own time/ activities
Behaviour Maladaptive behaviour; tantrums/meltdowns; aggression; self-injury Habit problems Sleep latency and waking; eating problems; toileting problems Learning School readiness; early literacy; early numeracy
Daily living skills Feeding self using cutlery; dressing self Global measure of
function
Global measure of outcome
Participation Social relations Sibling relationship; friendships; attending family events; attending birthday parties
Subjective well-being (quality of life)
Coping/resilience; self-esteem
Social inclusion Social participation; social exclusion; difficulty with attending appointments; awareness of danger
Family Measures Interaction style Synchrony; shared attention
Parenting Parent firm and fair; parent warmth to child
Parent stress Parent stress; parent coping style; parent anxiety and depression Family quality of life Impact on family; family cohesion
adopted was informed by discussion within the project team, and inspection of other relevant frameworks such as groupings of target symptoms/skills from interventions studies in ASD,20,27education outcomes,28
grouping of interventions by Research Autism (http://researchautism.net/pages/autism.treatments.therapies. interventions/) and compilation of measures for children with developmental disabilities.29One area
discussed was how to categorise quality of life, which is essentially a construct separate from the ICF-CY. The decision was made to include it in the participation domain, as it implies how an individual interacts with their environment.30For the MeASURe conceptual framework, there are four primary domains, with
subdomains in each.