entidades federativas Total 22 municipios

Participants

Diagnostic accuracy employed data from sample S4 (n=152) as this sample group self-reported dyspraxia.

Methods

The accuracy of a test to discriminate cases with or without a condition may be evaluated using a Receiver Operating Characteristic (ROC) curve analysis (Metz 1978; Zweig and Campbell 1993). In this study the ROC curve was employed to discriminate cases self-reporting or not self-reporting dyspraxia. The accuracy refers to the amount of agreement between the index test and the reference standard. The reference standard employed was the self-report of dyspraxia. The index test was the FDQ-9. The number of participants who self-reported dyspraxia (n=7). To establish the diagnostic accuracy of the ROC curve analysis the area under the curve (AUC) was assessed. A

perfect test will have an AUC of 1.0. Tests may be defined as >0.9 (high accuracy); 0.7-0.9 (moderate accuracy); 0.5-0.7 (low accuracy) (Swetts 1988).

In addition there was a requirement to establish a cut-off point at optimal sensitivity and specificity. Determining the optimal cut-off points was explored by employing two methods. The first assumes the best cut-off point for balancing sensitivity and specificity and is the point on the curve closest to the (0,1) point . In this method the minimal value for (1-senstivity)2 + (1-specificity)2 is the cut-off point (Perkins and Schisterman 2006). The second method is calculated using the Youden index (J). Where J = maximum (sensitivity + specificity -1) (Youden 1950; Perkins and Schisterman 2006).

Results

The area under the curve was 0.918 [95% CI 0.837 – 1.000] with a standard error of 0.042 (p < 0.001). This meant a randomly selected individual who self-reported dyspraxia would have a test score (FDS) higher than that of a randomly chosen individual who did not report dyspraxia 92% [95%CI 84% - 100%] of the time. This represents a diagnostic test with high accuracy (Swetts 1988) and may be viewed (See figure 4-10).

Figure 4-10 Receiver Operating Characteristic (ROC) curve using the total scores of the FDQ- 9 and those who self-reported dyspraxia. The sample group were S4 staff and students from a university (n = 152).

To calculate a cut-off score two methods were employed. The first method involved balancing sensitivity and specificity and involved finding a minimal value, the second involved calculating the Youden index maximum score. A minimal value of 0.055 [95%CI 0.023 – 0.100] was calculated in which sensitivity and specificity were balanced. The cut-off occurred with an FDS of 21.5. A review of the 95% CI meant that this cut-off score could range from 20.5 – 22.5. The sensitivity and

specificity of 20.5 would be 86% [95%CI 78% -94%] and 75% [95%CI 67% - 83%] respectively. The sensitivity and specificity of 22.5 would be 71% [95% CI 63% - 79%] and 88% [95% CI 80% - 96%] respectively. The Youden index maximum score was 0.671 [95%CI 0.589 – 0.753] and this occurred at a cut-off point of the FDS of 21.5. A review of the 95% CI indicated cut-off scores could range from 19.5 – 22.5. The sensitivity and specificity at a cut-off of 19.5 would be 100% [95%CI

92% - 100%] and 66% [95% CI 58% - 74%] respectively. The sensitivity and specificity at a cut-off score of 22.5 were recorded above.

The coordinates of the curve and associated sensitivity and 1 - specificity are presented (See table 4-12). Based on the two methods described above a cut-off score of FDS 21.5 were achieved. Based on a pragmatic approach as there are no half measures in relation to the FDS especially when used in the clinical setting the appropriate cut-off score would be an FDS of 22.

Table 4-12 Coordinates of the curve (Functional difficulties scores), Sensitivity and 1- Specificity

The sensitivity of a test relates to the proportion of individuals with a condition who are correctly identified by the test. In this case 86% [95% CI 78% - 94%] of those with a cut-off score of FDS 22 would be correctly identified by the test.

Functional difficulty scores Sensitivity 1-Specificity 10.0 1.000 1.000 11.5 1.000 0.986 12.5 1.000 0.945 13.5 1.000 0.897 14.5 1.000 0.814 15.5 1.000 0.731 16.5 1.000 0.648 17.5 1.000 0.510 18.5 1.000 0.407 19.5 1.000 0.338 20.5 0.857 0.255 21.5 0.857 0.186 22.5 0.714 0.117 23.5 0.571 0.076 24.5 0.571 0.055 25.5 0.571 0.041 26.5 0.571 0.028 27.5 0.571 0.014 29 0.286 0.007 31 0.000 0.000

The specificity is the proportion of the individuals without the condition who are correctly identified by the test. In this case 81% [95% CI 73% - 89%] of those with a cut off score of FDS 22 would be correctly identified as not having the condition and the graph may be viewed (See figure 4-11).

The results of the two methods described above suggested a cut-off score of FDS 22. This score achieved the sensitivity recommended by the APA (1985), but the specificity was less than that recommended.

Figure 4-11 Graph of sensitivity and specificity with a maximum Youden index in sample 4 (n=152)

The proportion of true positives, true negatives, false positives and false negatives employing a cut off score of 22 were presented in table 4-13.

Table 4-13 Proportion reporting a true positive, true negative result, false positive and false negative result when the cut off for the FDQ-9 was 22 (n=152)

Total score Dyspraxia No dyspraxia Total

≥ 22 6 28 34 < 22 1 117 118 Total 7 145 152 Prevalence = (7/152) x 100 = 4.6% [95%CI 0% - 13%] Sensitivity = 6/7 =86% [95% CI 78% - 94%] Specificity = 117/145 = 81% [95% CI 73% - 89%]

Positive predictive value = (6/34) x 100 = 18% [95%CI 10% - 26%] Negative predictive value = (117/118) x 100 = 99% [95% CI 91% - 100%]

The positive predictive value is the proportion of subjects with a positive test who are correctly diagnosed. However, this statistic is dependent on prevalence. The higher the prevalence of a condition in the group tested the higher the PPV. It is therefore appropriate to calculate the positive likelihood ratio which is independent of the prevalence calculation. The positive likelihood ratio indicates the odds of a condition increase when the test is positive. In this study this equated to (sensitivity / 1- specificity) = 4.61 [95% CI 3.93 – 5.38]. The likelihood ratio was high which suggests this test provides useful information (Petrie and Sabin 2005).

Exploring the results of the index test (FDS) and the reference standard (Dyspraxia)

It is suggested that to address the quality of the reporting of diagnostic accuracy researchers should consider the STARD checklist (Bossuyt 2003). This checklist is summarised in a table (appendix 22). In item 19 of the STARD the recommendation is to report a cross tabulation of the index test and the reference standard, this is presented in Figures 4-7 and 4-12. Presenting this data enabled the distribution of continuous data to be viewed and reported.

Figure 4-12 Bar chart showing the continuous results of the functional difficulties scores and the reference standard in Sample 4 (n=152). Self report dyspraxia (n=7) no self report dyspraxia (n=145)

The bar chart demonstrates the continuous distribution of FDS from participants in S4 (n=152).  FDS ranged from 11 - 30. FDS of 11 indicated being ‘very good’ at each of the 7/9 and

‘good’ at 2/9 items respectively.

 FDS of 30 indicate being ‘very poor’ at 3/9 items and ‘poor’ at 6/9 items respectively.  There was a trend for a higher percentage of participants who did not self-report dyspraxia

to have scores that span the range.

 Participants who did self-report dyspraxia tended to have higher scores indicating more functional difficulties. All participants who self-reported dyspraxia recorded FDS ≥20 indicating functional difficulties in 2/9 items or more.

 There were a number of participants who did not self-report dyspraxia who recorded high FDS. It is suggested that this group with high FDS may have had functional difficulties in childhood which continued into adulthood but were not assessed for dyspraxia/DCD in their early years. As mentioned previously (See 4.5.7) this is a group that has been previously identified and discussed (Kirby et al 2008).