In this section, we demonstrate how to calculate the posterior distribution for a Gaussian signal under the assumption of Gaussian noise and discuss the result.
Gaussianity for the signal means that we assume a Gaussian prior probability distribu- tion, given by P(s) =G(s, S) := 1 |2πS|1/2e −12s†S−1s . (1.37) Here, S =ss†P(s) (1.38)
is the signal covariance describing the size of the typical fluctuations in the signal and the correlation structure and the vertical lines denote a determinant. If the signal is just a single number, the covariance becomes the variance. The term in the exponential of the Gaussian can also be written explicitly as
s†S−1s= Z M dx Z M dy s¯(x)S−1(x, y)s(y). (1.39) In Eq. (1.37) we have also assumed that the prior is centered on zero, i.e. hsiP(s) = 0. Generalizing the calculation to a Gaussian prior with a different mean is straightforward. We also assume signal-independent Gaussian noise with zero mean, i.e.
P(n) = G(n, N), (1.40)
where the exponentiated term now reads
n†N−1n =X i X j ¯ ni N−1 ijnj, (1.41)
since the noise, like the data, is always finite-dimensional.
1If one thinks of s as a finite-dimensional vector, it becomes clear that ss† is a matrix ands†s is a
22 1. Introduction
The first step is to calculate the likelihood function, which under the assumption of Eq. (1.26) as data model, can be done easily by marginalizing over the noise,
P(d|s) = Z Dn P(d, n|s) = Z Dn P(d|n, s)G(n, N) = Z Dn δ(d−Rs−n)G(n, N) =G(d−Rs, N). (1.42) Here, theδ-distribution arises since the data are fully determined once the signal and noise are given. Next, we simply multiply the result with the signal prior to obtain the posterior
P(s|d)∝ P(d|s)P(s)∝ G(d−Rs, N)G(s, S)∝ G(s−m, D). (1.43) The last step involves a little algebra, after which one finds
D= S−1 +R†N−1R−1 (1.44)
for the posterior covariance and
m =DR†N−1d (1.45)
for the posterior mean. This last equation is known as the Wiener filter (e.g. Enßlin et al. 2009). Since the posterior is again Gaussian in this case, the posterior mean and maximum a posteriori solution agree and are both given by the Wiener filter.
The terms in the Wiener filter equation deserve a short discussion. Starting from the right side, the field R†N−1d = j is known as information source in the context of infor- mation field theory. It is a response-over-noise weighted representation of the data. The weighting scheme downweights data points that have been poorly observed and the ones for which the noise contribution is expected to be large. This field drives the reconstructed map m away from the zero mean of the prior.
Next is the application of the posterior covariance operator, D, called information propagator. The reconstructed map at one position, m(x), is made up of the values of the information source at all positions, each weighted with the appropriate value of the information propagator,
m(x) =
Z
M
dy D(x, y)j(y). (1.46)
Hence, the information is propagated from each position y to its final destinationx. The information propagator, given in Eq. (1.44), consists of two parts. One part is a projection of the inverse noise covariance into the space of possible signals by means of the response operator, R†N−1R. Qualitatively, this operator distinguishes between regions of
M that are well observed and those which are not. The other part of D is the inverse signal covariance, S−1, which separates regions of high expected signal fluctuations from those with low variations. In the limiting case of a region with high signal fluctuations that are well observed, S−1 ≈ 0 and R†N−1R 6= 0, the information propagator becomes D ≈
R†N−1R−1
, thus effectively undoing the response-over-noise weighting of the data and applying something close to an inverse response operator to compensate for the response
1.2 Signal inference 23
operator in Eq. (1.26), without extrapolations. In the other extreme case, in which the observations are very poor, R†N−1R ≈ 0, the information propagator is the same as the signal covariance and extrapolates widely. In general, there will be poorly observed regions, for which information has to be propagated across longer distances, and well observed regions, for which little extrapolation is necessary. The information propagator automatically balances the information from all locations in the appropriate manner for each point of the reconstruction. Note, however, that the two parts of the propagator will in general have different eigenbases, so one always has to consider the whole of M and cannot consider points or regions individually. The Wiener filter theory forms the basis for the signal inference techniques that we develop in Chapters 3 and 5.
It should be clear by now, that the signal and noise covariances play an important role in the reconstruction of Gaussian signals. Their precise knowledge is important for an accurate result. However, in general, at least the signal covariance S is not known. To overcome this problem, several strategies have been developed (Wandelt et al. 2004; Jasche et al. 2010b; Enßlin & Weig 2010; Enßlin & Frommert 2011). We will briefly outline the derivation as done by Enßlin & Weig (2010), since Chapters 3 and 5 build on it directly.
The basic idea is to augment the Gaussian signal prior with a well-chosen prior for the signal covariance,P(S), and derive the posterior via marginalization over the covariance,
P(s|d)∝ P(d|s)P(s) =P(d|s)
Z
DS P(s|S)P(S). (1.47) The resulting posterior, however, is highly non-Gaussian and calculating its mean is there- fore not possible analytically. Enßlin & Weig (2010) therefore suggest to find an optimal Gaussian approximation to this posterior, described by its mean m and its covariance D. Fitting formandDthen gives the critical filter estimate for the signal and its uncertainty. The detailed assumptions regarding the structure of the signal covariance, the choice of priors, and the approximations made can be found e.g. in Chapter 3.