La educación producto del proceso de Enseñanza/Aprendizaje La educación que resulta del proceso E/A puede interpretarse en

Once we have obtained our data, either AD or IPD, we need to choose the model to use. There are several meta-analysis models available; here our focus is on the distinction between one-stage and two-stage models.

Broadly, one-stage models correspond to the IPD-MA and two-stage models to AD-MA; how- ever, while the former can not be used when in possession of the AD only, the latter are often used even with IPD.

In general,we talk about one-stage analysis when all the IPD from the contributing studies are modelled simultaneously in a single step; this is not done by simply running the desired model on a ‘huge’ dataset composed of all the IPD, a practice that can and usually does lead to invalid inferences (Abo-Zaid et al., 2013). Methods taking into account correlation between observations coming from the same study must be used instead, making it possible to control for possible sources of heterogeneity between the studies. One possible solution is simply to include study ID in the model as a covariate with a fixed effect. Otherwise we can use methods allowing for random effects for clusters; these are the so-called multilevel models (Goldstein, 2011), already known as hierarchical or mixed models.

Coming back to our simple example (2.1.1), a simple hierarchical model could be:

yi,s = (β0+ u0,s) + β1xi,s+ i,s (2.3.1)

Here, the only substantial difference with respect to model (2.1.1) is given by the new term u0,s, that is a study-specific random intercept that can be assumed to follow a normal distribution N (0, σu). If we believe that not only the intercept, but even the slopes should be modelled allowing for a different slope in each constituent study, then we might opt for a random intercept and slope model:

The random slope term u1,s can be modelled in different ways, for example a simple option is to suppose that u0,sand u1,sfollow a bivariate normal distribution, either with an unstructured covariance matrix or with correlations set to zero.

Since in IPD-MA, variances of the residuals are usually different in the different studies, it is particularly important considering models for complex level 1 and 2 variation in this setting (Goldstein, 2014). For example, the assumption in model (2.3.2) that i,s ∼ N (0, σe2) could be relaxed by allowing for study-specific residual variances, i,s ∼ N (0, σe,s2 ).

Regarding the estimation method, maximum likelihood is known to lead to biased estimates of the variance components, and therefore Restricted Maximum Likelihood is often used instead, particularly with smaller sample sizes.

When we use a two-stage model, instead, in the first step we fit our analysis model of interest, i.e. (2.1.1), within each study if we have the IPD, or we collect the results, i.e. β1,sˆ and associated standard errors ˆσ1,s, from the single studies s if we only have the AD, and then we synthesize the evidence in order to get some more definitive conclusions about the treatment or exposure effect of interest. A natural idea is to weight the results from the single studies according to the magnitude of the standard errors; this is what is done in the Inverse Variance Weighting (IVW) approach (DerSimonian and Laird, 1986).

There are two different approaches to IVW: a fixed-effect analysis or a random-effects analysis. In both cases, we weight the estimates for the single parameters coming from the different studies according to the inverse of their variances. The difference between the two methods is that while in the former we assume a common treatment effect across the different studies, in

the second we assume that the actual effect within each study is only a random draw from a certain distribution; therefore we make an adjustment to the study weights according to the extent of variation, or heterogeneity, among the varying intervention effects. Given estimates

ˆ

β1,s and ˆσ1,s, the fixed effect IVW model is:

ˆ

β1,s= β1+ s, s ∼ N (0, σ1,s2 ) (2.3.3)

The maximum likelihood estimate ˆβ1,F E under this model is:

ˆ β1,F E = PS s=1 ˆ β1,s ˆ σ2 1,s PS s=1 1 ˆ σ2 1,s = PS s=1wsβˆ1,s PS s=1ws (2.3.4)

Here ws are the weights we give to the estimates and, in the case of IVW, they are just equal to the inverse of the variance for each study, i.e. ws = _σˆ12

1,s

. Note that within-study variances ˆ

σ2

1,s are assumed to be known. An estimate of the variance of ˆβ1,F E is:

ˆ V ar( ˆβ1,F E) = 1 PS s=1ws . (2.3.5)

The corresponding random effects model is:

ˆ

where us and s are independent. The us are not intrinsically associated to each study s, but if we were able to re-run any study s we could draw a different value us for the same study(i.e. exchangeability). The second addition of this model, τ2, is the between-study component of variance. The algebraic form of random effects estimates of β1 and σ1 are really similar to (2.3.4) and (2.3.5), with the only difference that the weights ws are substituted by ws∗, with:

w_s∗ = 1 ˆ σ2

1,s+ ˆτ2

(2.3.7) There are many different methods available in the literature for finding an estimate ˆτ2 of the between-study component of variance τ2, among which the most used is that proposed by Der Simonian and Laird (DerSimonian and Laird, 1986); however, in some settings this has been criticised (Hardy and Thompson, 1996; Brockwell and Gordon, 2001) and many other methods have been developed (Hunter and Schmidt, 1990; Sidik and Jonkman, 2005; DerSimonian and Kacker, 2007). The main problem with the Der-Simonian and Laird method is that often standard errors of the estimates are underestimated; therefore (Hartung and Knapp, 2001) proposed a modified estimate of the variance, and also using a t distribution for deriving confidence intervals in place of the standard normal distribution.

As we already said, with AD, only two-stage MA is possible but, as for the IPD case, we can still choose between a fixed-effect or a random-effects analysis. This is usually done both by means of appropriate strategies, like the examination of the Q and the I2 _{statistics (Higgins} and Thompson, 2002; Higgins et al., 2003), and by an appropriate consideration of expertise knowledge in the area of the meta-analysis (Borenstein et al., 2010).

On the other hand, with IPD, we also need to choose between one-stage or two-stage analysis. It has been generally thought that both methods lead to similar results and conclusions (Stewart et al., 2012); however recently (Debray et al., 2013) showed how the two methods can in principle lead to different conclusions in some situations and they recommend to use one-stage models, particularly in situations where few studies or few individuals per study are present.