This chapter forms a basis for a paper published in Statistics in Medicine in 2008, in collaboration with J.B. Copas [20]. A copy is given in Appendix A1 (page 139 onwards), for which some additional technical details and another numerical exam- ple for the purposes of illustration can be found. Modelling publication bias in a meta-analysis requires making some assumptions about the selection process, other- wise inference is impossible. The downside to making such assumptions is that it is not possible to verify their validity. In this chapter, we presented a robust non- parametric method which aimed to relax the assumptions about the selection process. The surely plausible and generally widely accepted idea that we adopt here is that selection depends in some unspecified way on a study’s P-value.
Two approaches to providing a P-value were presented: the first being the permuta- tion P-value, which was based on standard permutation theory. The basic concept was to permutate the yi values such that different values of ˆθ were generated. In
practice, taking a sufficiently large sample, the proportion of these different ˆθ values that were greater than or equal to the observed value of ˆθ was our P-value. The sec- ond approach involved an approximation P-value, which resulted in having quite a simple form, depending on the number of studies in the meta-analysis and the sample correlation of the radial plot.
The concept behind these two approaches is quite elegantly simple. However this trade-off for simplicity and avoiding making strong assumptions about selection comes in the form of loss of power. The published paper as shown in Appendix A1 develops theory about the power functions relating to the approximate robust P-value method, and compares this to the power function from the conventional fixed effects model (which clearly makes more assumptions about selection). There is an inevitable loss of power with the robust P-value, for which the severity of this loss depends mainly on γ, the coefficient of variation of the observed vi, defined as γ = sv/v¯. A small
value of γ implies there is a small spread of values along the x-axis of a radial plot, and in these scenarios, the loss of precision is very large. For larger values of γ, the loss of power is still present but less significant.
We conclude this chapter by mentioning that the paper in Appendix A1 includes a different numerical example - namely a meta-analysis of randomised controlled trials of intravenous streptokinase in the prevention of death following myocardial infarction. This example is discussed in the text edited by Egger et al. [32] and the data can be found there. Whilst providing an opportunity to present the robust P-value method on a different data set, this meta-analysis is a good example of a dataset where we have a larger value of γ (because we mainly have small studies and a couple of large studies), resulting in the robust P-value still having a loss of power, but not as much as the passive smoking dataset (which had a smaller γ). Again, full details can be found in the Appendix.
5
Applications of Parametric Selection Functions
in Meta-Analysis
5.1
Introduction
It is becoming increasingly recognised that standard methods in meta-analysis can produce potentially misleading results if certain issues are not addressed. Two such issues are heterogeneity and publication bias. Heterogeneity refers to variation be- tween studies within a meta-analysis that can not be fully explained by just sampling error alone. It may be inappropriate to combine study results if the studies are not estimating the same quantity of interest.
The issue of publication bias is discussed here. Publication bias is caused by simply assuming that studies within a meta-analysis constitute a random sample of studies from some population of interest. The shared belief is that studies with statistically significant reviews are more likely to be submitted for publication than those with non-significant results [29]. This non-random sampling that is taking place will there- fore create bias and in turn pose a serious threat to the validity of the results of the meta-analysis.
As reviewed in Chapter 2, various approaches have attempted to model publication bias in meta-analysis, as reviewed by Sutton et al. [80]. One such approach is the use of selection functions. Hedges [43] first introduced selection functions into meta- analysis. Essentially, selection functions model the probability that a study is selected for publication, usually determined by the study’s P-value. There are many exam-
ples in the literature of the selection functions taking some kind of parametric form. There usually is an adjustable parameter, β, that models the selection. Since the selection process is unknown, and therefore we know nothing about the value of β, we consider a sensible range of different values of β as part of a sensitivity analysis to investigate how our inferences change.
Following on from Copas and Jackson [16], the selection function takes the form
P(selection|y, σ) = a(y, σ), (38)
for some function a(y, σ), where it will be assumed that y ∼ N(θ, σ2). We let p,
the (unknown) overall selection probability, be defined as p=E[a(y, σ)], expectation being over a population (y, σ) of studies. With this definition of p, and by assuming that a(y, σ) has a parametric form, we will be able to directly assess the likelihood function, and make inferences about a bias-corrected θ using a maximum likelihood approach. From this, we will be able to conduct hypothesis tests, and explore datasets by considering likelihood contours. The crucial argument here is that we want a sen- sitivity analysis for different fixed values ofpsince it is impossible to estimatea(y, σ).
In Section 5.2, a description will be given of the maximum likelihood approach with parametric selection functions. Section 5.3 will focus on selection functions where the adjustable parameter is scalar. Section 5.4 will briefly review the Heckman-type se- lection model, and re-examine the methods used by Copas and Shi [21]-[23]. Finally, Section 5.5 analyses the effectiveness of the bound for confidence intervals proposed by Henmi et al. [47] by comparing the confidence intervals for θ, when the selection functions are assumed, and when the Bounds method is used.
Throughout, two examples will be discussed. The first example will be the passive smoking dataset used by Hackshaw et al. [40] concerning the relationship between passive smoking and lung cancer. We include a second example to demonstrate the methods used - a dataset that has not yet been discussed in this thesis. The data relates to the effectiveness of prophylactic corticosteroids, an example which was first
discussed in Copas and Jackson [16].