Los Riesgos de la Concha

In the sensitive area of drug use, survey responses may be particularly vulnerable to aspects of survey design such as sample selection, the phrasing of the questions, interview conditions, and the perceived confidentiality of the answers. If survey data are to produce credible measurements of drug use, researchers need to be confident that the answers are not just an artefact of the particular survey chosen. And if surveys do differ, they need to know by how much and, ideally, why. A necessary first step in exploiting survey data is therefore to assess the practical effect of differences in design amongst commonly used surveys.

In this section the authors have compared the data sources which they used in this study – the OCJS, SS (both 2003) and AS (2003-04) – with each other where appropriate, and with two other similar surveys, the BCS (2002/3) and NEW-ADAM (1999-2002). In a fuller study (Badillo et al., 2005), the authors have also compared the 1998 releases of the BCS and SS with the YLS, and the authors refer briefly to these results in this section. To facilitate comparisons, they have focused on two simple consumption measures of the six drugs considered: whether respondents reported ever taking each drug, and whether they had done so in the last year. The questions required to construct these measures appear in all the surveys and also have the advantage that they are generally more straightforward and unambiguous than questions about quantities purchased and consumed. The first measure is a useful benchmark for past drug use, whilst the second measure is a better reflection of “current” drug use and should also be less affected by recall error. This approach has been to compare subgroups of respondents who are as similar as possible across the different surveys (specifically, the same groups surveyed in the same year in the same geographical area). The data have also been weighted where possible to adjust for differing sample composition.

The 2003 OCJS and 2002/3 BCS

This subsection begins with a comparison of the 2003 OCJS, with another general population survey, the 2002/3 BCS.22 Both cover private household in England and Wales, but in contrast to the BCS, the OCJS is less concerned with experience as a victim of crime and more with offending behaviour by the respondent. Both surveys use CASI and have similar responses rate (about 75%). However, question wording and routing differ somewhat. Whilst the question asking about ever taking drugs is similar in the two surveys, the BCS then asks if the respondent has taken that drug in the last 12 months whilst the OCJS splits the time period up into the last four weeks and then the last 12 months (not including the last four weeks).

As shown in appendix Table A5.1, the OCJS elicits a higher proportion of drug users than the BCS. For example, the weighted estimates show that 34.4 per cent of respondents in the OCJS reported ever having taken cannabis, compared to 30.6 per cent in the BCS. This is an absolute gap of 3.8 percentage points and a proportionate gap of 13 per cent. There are similar proportionate gaps for the other drugs, with, in some cases, larger proportionate gaps when the measured prevalence is low (for example, 0.2% reported taking crack in the last

22

The OCJS took place in the first half of 2003 whilst the BCS was conducted on a monthly basis throughout the 2002 Financial Year and so, whilst the two surveys have an overlapping period of time, they are not concurrent.

year in the BCS, compared to 0.3% in the OCJS). However, these differences are not dramatic and, for three drugs (amphetamines, crack and heroin), they are not statistically significant. In Badillo et al. (2005) the authors compare 16- to 24-year-olds from the BCS and OCJS youth boosts. The OCJS, once again, suggests higher prevalence and a higher level of recent drug use than the BCS. The authors find similar proportionate differences, and, as for the full sample, no significant evidence of a difference for amphetamines, crack and heroin. In Badillo et al. (2005) the authors also analyse the sub-sample of interviewees who responded to both the BCS and the YLS. With the caveat that respondents were on average eight months older in the YLS than they had been in the BCS, the expectation would be to see very similar responses from these identical samples. In fact, the results show differences of similar magnitude to those above. This suggests that the gaps between the surveys are likely to be due to survey and interview factors rather than the sampling of respondents.

The 2003 SS and 2003 OCJS

The SS sample of young people (aged 11 to 15) in England with English 11- to 15-year-olds from the OCJS are now compared. The main difference between the surveys was the interview mode: a paper-based self-completion survey conducted under “exam conditions” in the SS, compared to CASI self-completion in the OCJS.

The SS elicits a much higher proportion of respondents who claim to have ever used drugs in their life (see appendix Table A5.2). Whilst almost eight per cent of those questioned in the OCJS say that they have tried cannabis, 15.3 per cent give the same response in the SS. This is the largest absolute difference. The biggest proportionate difference is for heroin: whilst just 0.1 per cent of the OCJS sample in England aged between 11 and 15 say that they have tried heroin before, the proportion in the SS is 1.1 per cent. There is a similar pattern to reports of drug use in the past 12 months, for example the proportion who have used cannabis in the last 12 months in the OCJS (6.2%) is less than half of that elicited in the SS (12.5%).

It is hard to say for certain which set of figures is accurate but, given that the OCJS is conducted in the student’s home whilst the SS is conducted in a more anonymous classroom setting, it seems plausible that the SS context encourages respondents to be more honest in their answers. Some evidence consistent with this was found when the analysis was repeated using only those respondents in the OCJS who completed the questionnaire with no other household member present in the room. The proportion reporting some use of cannabis increased to 10.6 per cent and the proportion reporting amphetamine use increased to 0.7 per cent.23 This narrows the gap between the OCJS and the SS, but the estimates are still significantly different. An alternative (and, in the authors’ view, generally less plausible) interpretation of the SS-OCJS difference is that proximity to friends and peers in the classroom in the SS may lead to ‘bragging’, despite the fact that other pupils cannot see the answers of the respondent. This seems most likely in the case of heroin, where the prevalence figure is higher than one might expect.

NEW-ADAM and the Arrestee Survey

Finally, the Arrestee Survey (2003-04) is compared with its predecessor, NEW-ADAM (1999- 2002). Because they cover different years and were designed differently, the estimates are expected to differ more than in the previous comparisons. The results (see appendix Table A5.3) confirm that there are some quite large differences between NEW-ADAM and the AS. The ‘ever taken’ prevalence of amphetamines is 13 percentage points lower in the AS than in NEW-ADAM and the proportion having taken cannabis in the last year is ten percentage points lower. These differences may be due in part to changes in demand (there is strong evidence of a downward trend in amphetamine use) and in police arrest practices. Other estimates are very similar across the two surveys: for example, the proportion of respondents ever having used crack is about 36 per cent in both. To investigate whether the observed differences may stem from the different geographical coverage and time periods of the

23

For the less common drugs, there are too few observations to make reliable inferences in the restricted OCJS sample.

surveys, the authors restricted NEW-ADAM data to its most recent year before the AS, and only used police suites common to both NEW-ADAM and the AS. However, the discrepancies between the two surveys are not obviously smaller in the restricted comparison. A caveat is that the estimates in the smaller samples are less precisely estimated, and that the surveys are still separated by two years, during which drug consumption could have changed.

Overall, larger proportionate differences are found between the two surveys of arrestees than between the general population surveys (OCJS and BCS). Much of the difference is likely to be caused by the very different survey designs and survey periods.