Huggins (1993a, b) defines robust maximum likelihood estimates as the estimates maximising the following modified version of (1.2):
- ± Pj(-j) ^ (ki/^loglV jl (3.1)
j=l
■1/2
where Zj = Vj (yj - Xja) and {pj} are suitable non-negative functions. The zj are
u i «
vectors, not scalars: to handle this, we will put pj(zj) = ^ p(zjk) and \|/j(zj) = k=l
(\j/(zji), ..., y (zjmj))T, where p(zjk) and y(zjk) are suitable scalar-valued functions. To ensure consistency at the normal model, ki = E[\j/(e)e] where e ~ N(0, 1). The robust maximum likelihood estimator of 0oi, 1 < i < c, found by maximising (3.1) with respect to 0i, satisfies the equations
(1/2)Z z[ vj 1/2[ZiZ^]jV-1/2Vj(zj) - tr[KijV.1 [Z .Z ^] = 0. (3.2)
j = i
where Ky = kilmj, or, more generally, Kij = E[ej\|/j(ej)T] with ej ~ N(0, Imj). Appendix 2 contains details on the derivative of V-1/2 which appears in (3.2).
A popular choice for p is
P(Zjk) = | (1/ 2 )zjk lzJk l - K
[K|zjk| - (1/2)k2 otherwise
which is the Huber p function with tuning constant K. Its derivative, which is bounded, is
V(zjk) = {Zjk Ik sgn zjk
M 2 K
otherwise.
However, to ensure robustness using (3.2) we require not only that \|/j(zj) be
T
bounded, but also that z- vj/j(zj) be bounded. A bounded p function leading to a redescending \|/ function fulfils these conditions: Huggins (1993a, b) uses Tukey's biweight function which is
p(z.k) = |( 1 /6k2)((1 - (zjk/K)2)3 - 1) |zjk| < K
J [0 otherwise
with derivative
¥(zik) = | Zjk( 1 - (Zjk/K)2)2 |zjk| < K
J [0 otherwise.
The estimating equations are then susceptible to multiple solutions, but Kent and Tyler (1991) give conditions for the uniqueness of estimators involving redescending \\f
functions. To avoid having to choose a redescending \|/ to achieve robustness, we may estimate 0oi, 1 < i < c, by solving the following estimating equation:
8
(1 /2 )X Vj(zj)Tv'1/2 [ZjZ^ljVj 2\j/j(zj) - tr[K2jV“ ' [Z .Z ^] = 0, (3.3)
j=l
where K2j = E[\|/j(ej)\|/j(ej)T] with ej ~ N(0, Imj).
The robust maximum likelihood estimator of 0Lq, found by maximising (3.1)
£ x ] ' v - |/2Vj(V rl/2(yj - X ja » = 0. (3.4)
j=l
We call the procedure of solving (3.2) and (3.4) Robust ML I. We call the procedure of solving (3.3) and (3.4) Robust ML II, since (3.3) can be viewed as a generalisation of Huber's (1964) Proposal 2 for robust regression with unknown scale parameter. It appears that there is no single likelihood function that has (3.3) and (3.4) as its derivatives with respect to 0 and a, but this is not necessarily a drawback because we gain flexibility by having a set of estimating equations that are not tied to a single likelihood.
This situation is clearly related to the situation studied in the large body of work on estimating equations, also called estimating functions: see Godambe and Kale (1991) for an recently-written introduction to the topic. Rather than proposing an estimator directly, they consider an equation whose solution forms the estimator. The most important conditions on the equation are that it has zero expectation and finite variance. For a connection to robust estimation, see Godambe and Thompson (1984) who set M-estimation of a location parameter into an estimating function framework. Godambe and Kale also discuss the relationship between estimating equations and quasi-likelihood. In the latter situation, there is insufficient information to construct a likelihood, but there is enough information to write down a mean and variance structure e.g. Ey = X a and Var(y) = V(0). A set of estimating equations is then constructed that have the same properties as the derivative of a log-likelihood, although the log-likelihood does not necesarily exist. In some cases it does exist, and McCullagh and Neider (1989, sec. 9.3) give conditions for its existence when the observations are dependent.
We considered a range of possibilities for defining robust REML estimators based on the REML estimating equations, and below we present two proposals, denoted Robust REML I and n. One of the advantages of classical REML over maximum likelihood is that REML incudes an adjustment to allow for the loss of degrees of freedom incurred in estimating the fixed effects. One of the reasons for robustifying REML is to combine this adjustment with a bound on the influence of outlying observations on the estimate. We take the REML estimating equations (1.6)
and apply a two-stage robustification process to arrive at estimating equations analogous to (3.2) - (3.3). The first stage of robustification is to replace a(0) = (XTV-1X)-1XTV-1y in (1.6) with ocr(0), a robust estimate of a. The second stage is to apply \\f functions and consistency corrections. Thus our two proposals for robust REML estimation of 0oi, 1 < i < c, are:
Robust REML I. Solve
(l/2){(y - X aR(e))T v -'Z iZ jv -l/2 V(V-i/2(y - X aR(0)))
-tr[K iPZjZT]) = 0. (3.5)
Robust REML II. Solve
( l / 2 ) ( V(V -l/2(y - XaR(e)))TV-l/2ZiZTj V -l/2 ¥ (V -l/2(y - X aR(0)))
- tr[K2PZiZ7]) = 0. (3.6)
Then, given an estimate 0 of 0O using (3.5) or (3.6), 0to may be estimated by, for example, robust weighted least squares, which involves minimising
Using Robust REML I or II increases the number of parameters requiring iterative estimation compared to REML, because ocr(0) requires iterative estimation while an explicit formula exists for a(0). However the difficulty of estimation is not increased because we do not attempt to solve (3.5) and (3.7) or (3.6) and (3.7)
simultaneously, but we solve them separately in a two-step procedure. Given an initial value of 0, we solve (3.7) to obtain a value of (Xr(0). We then substitute this value of ocr(0) into (3.5) or (3.6) to obtain an updated value of 0. These estimates are used to solve (3.7) again and so on until both sets of parameter estimates converge.
We now return to non-robust estimation of variance components briefly, in order to prepare for showing that Fellner's method is the solution of a robust version of (1.6), in the manner of (3.5) - (3.6). Harville's (1977) algorithm for solving (1.6) basically consists of manipulating (1.6) into the form 0 = g(0), and then calculating successive approximations to 0 by taking 0new = g(0old). Laird (1982) points out that Harville's algorithm is equivalent to the EM algorithm of Dempster et al. (1977). The EM algorithm itself is a general method of maximising the likelihood of a set of data
L
Pj(v j 1/2 <yj - Xj«R(0)))j=i
for ocr(0). The solution also satisfies the estimating equation
(3.7) j=l