Microscopía túnel de barrido en sección trans-

The synthesis of the results of the studies included in the review serves three purposes:

a) To provide an estimate of the average effect of the intervention in question.

b) To investigate whether the treatment effect was similar for different studies, different settings and different populations.

c) To examine the differences in effectiveness of the intervention from different studies should they arise and the reasons for these.

Systematic reviews pool the data from the included studies in order to provide some overall conclusions. A qualitative or quantitative approach can be taken. However, one is advised that if the data are sparse, certain data are missing or the studies are too heterogeneous, a qualitative approach should be applied. A qualitative summary of the

analysis can be performed. Over the years, quantitative synthesis has attracted a great deal of attention, leaving qualitative synthesis in the background. Yet it should be noted that the quantitative phase of the analysis is an extension of the qualitative work where the homogeneity o f the studies is established.

Qualitative synthesis focuses on analysis o f the studies’ findings in terms of the subjects involved in each intervention, the setting, the outcome measures recorded and the environmental or modifying factors which may have influenced the results. Formal methods for qualitative analysis need to be developed as are presently available for quantitative analysis. This may bring with it a greater sense of credibility for qualitative synthesis.

Quantitative synthesis, on the other hand, pools estimates of treatment effects for each study. Presenting the effects of the studies in graphical form allows the overall picture to be seen more clearly. The treatment effects of each study are represented with diamond symbols horizontally. The size of the symbol reflects the weight given to the study in the analysis. The more reliable and informative studies are given more weight (CRD, January

1996).

Meta-analysis is the name given to the pooling of results, weighting the studies appropriately. Two models can be applied to the data when carrying out a systematic review (CRD, January 1996). First, the random effect model is the more conservative option which tries to assure that studies included in the review are representative of a

random distribution of the treatment effects. Taking into account between study variation, this statistical model tries to account for other factors which may have influenced the treatment effects. Its lack of confidence in the sampling techniques is reflected by the wide confidence intervals. Second, the fixed effect model ignores the between study heterogeneity and treats the estimates of treatment effects as a single score at the basis of each study. The model changes according to the data being considered. When dealing with measurements, all of which use the same scale, the estimate of treatment effect can be calculated from the weighted mean of the treatment effects of individual studies. If the data does not rely on the same scale, the mean effect of each study must be converted to a standardised value. Both models offer similar findings but must be used cautiously as assumptions are made about the findings in both cases.

Professor Hans Eysenck voiced his reservations regarding the pooling of data as described above, feeling that it was strange to merge the results of studies which were carried out independently, using different subjects in different settings, investigating different outcomes. The meta-analysis is far from perfect, as cases have been known where beneficial effects have been found which were later found to be false. This is reason to be cautious and to apply a test of homogeneity to one’s study. Such a test determines whether the variation between studies is likely to have arisen solely by chance or whether there are factors accounting for some of these differences which need to be explained. An insignificant result would demonstrate homogeneity. A test of

homogeneity will ensure that the lower confidence interval o f each study falls below the higher level of any trial. Heterogeneity would otherwise be evident. However, as the

precision and power of these tests is low, differences could still be present which need to be uncovered. Due to the imprecision of these tests, one must be cautious when interpreting the results (Greenhalgh, 1997).

To account for the variable validity of the studies, one must consider the inevitable biases. By synthesising results of different study designs and comparing the overall estimates, a comparison can be made. Alternatively, the studies can be cumulatively synthesised, noting any changes in the estimates of treatment effects as weaker designs are added. A meta-regression model could likewise be used to monitor any change resulting from the mixed validity and methodology of studies. In this case, strength of evidence can act as a variable. To assess the effect of validity on the overall results, sub­ analyses can be applied. For example, by extracting one or two dominant studies from the results, one can examine whether the overall results are similar for smaller, less powerful studies.

Due to the emphasis placed on quantitative and statistical aspects of research, the published literature can be subject to bias. Journals and the authors who submit papers focus heavily on the significance of the findings, in both the clinical and statistical sense of the word. In comparison with studies showing significant findings there are few studies published which reveal an insignificant effect (Chalmers & Altman, 1995). As the exclusion of this research can influence the overall conclusions of a review of this type, tests have been developed by which the possibility of publication bias can be assessed. The funnel plot, as it has come to be known, indicates the effect size of each study

according to the sample size (Greenhalgh, 1997). A funnel shape tends to appear as the variability in effect sizes alters as the size o f the sample decreases. If the funnel shows an incomplete shape this suggests that studies are missing and that publication bias may be affecting the results. Missing studies more often than not are small studies that show no effect. To minimise the possibility o f publication bias encroaching upon the results, a comprehensive search should be carried out. Specialised registers of trials should be referred to, as all studies are entered before the results are apparent and therefore present an unbiased account of the literature and, finally, funnel plots can be performed.

The synthesis and analysis o f the results marks the completion of the review itself but the most effective means of disseminating the information must be carefully considered. The Cochrane Collaboration play a role in this area as well, for the information becoming available as a result of systematic reviews needs to be suitably disseminated to the relevant parties in order for the purpose of the review to be fulfilled. The Cochrane Collaboration has several databases where reviews can be accessed and where the material is updated on a regular basis. For non-Cochrane reviews, the problem of journal space remains. Chalmers & Altman (1995) stress the importance of dissemination and encourage the development of electronic journals as a way o f addressing this problem.

CHAPTER II