Cronograma de actividades en contexto

3.6 Noise that Is Not White

White noise can be considered a mathematical abstraction that keeps our analysis (and our algorithms) simple. But this is not a fair statement; in many applications the noise is such that, to a good approximation, it can be considered white.

We have already encountered one definition of a white-noise vector, namely, that the covariance matrix is a scaled identity matrix. Another definition, more closely related to signal processing, is that all frequencies present in the time series signal e must have the same probability. The name “white noise” derives from the analogy with “white light,” in which all colors (i.e., frequencies of light) are present.

To state the above more rigorously, we need the m×m discrete Fourier matrix F (which is complex and symmetric) defined such that the vector e and its discrete Fourier transform ˆe are related by

ˆ

e =fft(e) = conj(F ) e, e =ifft(ˆe) = m⁻¹F ˆe

and satisfying conj(F ) F = m I. Then the vector e is white noise if the covariance matrix for its discrete Fourier transform is a scaled identity. The two definitions are obviously equivalent, because

Cov(ˆe) = Cov (conj(F ) e)) = conj(F ) Cov(e) F = m η²I,

but the requirement to ˆe is sometimes more practical. For example, we can say that a signal is “white-noise–like” if Cov(ˆe) is close to a scaled identity matrix, and hence its spectral components are nearly uncorrelated and nearly have the same variance. (But note that Cov(ˆe) does not imply that that Cov(e) is close to a scaled identity!)

3.6.1 Signal-Correlated Noise

To illustrate the usefulness of the above definition, we consider noise whose amplitude is proportional to the pure data. Specifically we consider the case where the noisy right-hand side is given by

b = b^exact+ eb with eb= diag(b^exact) e, (3.22) where e is a white-noise vector. In other words, the noise vector ebhas elements given by (eb)i= b^exact_i · ei for i = 1, . . . , m, and its covariance matrix is

Cov(eb) = diag(b^exact) Cov(e) diag(b^exact) = η²diag(b^exact)².

We will now justify that this type of noise is "white-noise–like.” That is, we must show that the covariance matrix for the discrete Fourier transform ˆeb= conj(F ) eb,

Cov(ˆeb) = η²conj(F ) diag(b^exact)²F, is close to a scaled identity matrix.

A standard result for discrete convolution (see, e.g., [35]) says that a matrix of the form conj(F ) diag(d ) F is circulant and its first column is given by ˆd = conj(F ) d .

Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

44 Chapter 3. Getting to Business: Discretizations of Linear Inverse Problems

Figure 3.11. When the noise amplitude is proportional to the signal; case study with shaw test problem. Top: The vectors b^exact, d = ((b^exact)²₁, . . . , (b^exact)²_m)^T, e, and eb. Bottom: Fourier spectra for b^exact, d , e, and eb.

Hence, by identifying d = !

(b^exact)²₁, . . . , (b^exact)²_m"T

we conclude that Cov(ˆeb) is circulant and its first column is

!Cov(ˆeb)"

:,1= η²conj(F )!

(b^exact)²₁, . . . , (b^exact)²_m"T

= η²d .ˆ

That is, the first column is the discrete Fourier transform of the vector d whose elements are the squares of those of the exact right-hand side b^exact. Like b^exact, the vector d is also dominated by low-frequency components, and hence the largest elements of ˆd are located at the top. Hence, Cov(ˆeb) has large identical elements along its main diagonal, and they decay in magnitude away from the diagonal. Consequently we can say that ebis “white-noise–like.”

Figures 3.11 and 3.12 illustrate the above analysis for the test problem shaw from Regularization Tools. In particular, note that both b^exact and d are dominated by low-frequency components, and that the spectra for ebresembles that of e; i.e., it is flat, and therefore e_b is “white-noise–like.”

3.6.2 Poisson Noise

Poisson noise often arises in connection with data that consist of counts of some quantity, such as light intensity which is proportional to the number of photons hitting the measurement device. We note that such data always consist of nonnegative integers.

To motive our notation for Poisson noise, consider the following alternative way to write data with additive Gaussian white noise with standard deviation η:

b_i ∼ N (b^exacti , η²), i = 1, . . . , m.

Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

3.6. Noise that Is Not White 45

| Cov(fft(e

b) )|

10 20 30 40 50

10

20

30

40

50 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Figure 3.12. The covariance matrix Cov(ˆeb) for the Fourier transform of the noise vector eb is close to a scaled identity matrix.

That is, the i th data value bi has mean valueE(bi) = b^exact_i and varianceV(bi) = η². Similarly, the data considered in (3.22) can be written as

bi ∼ N!

b^exact_i , (ηb_i^exact)²"

, i = 1, . . . , m,

which implies thatE(bi) = b_i^exact andV(bi) = (ηb^exact_i )²; i.e., the standard deviation is proportional to|b^exact_i | with proportionality factor η.

For Poisson noise, we recall that the Poisson distribution depends on a single parameter which is both the mean and the variance. Hence, data that follow a Poisson distribution¹ can be written as

b_i∼ P(b^exacti ), i = 1, . . . , m.

Thus, the mean and the variance of the i th data value bi are now given by E(bi) = b^exact_i and V(bi) = b^exact_i . This is clearly another example of noise that is correlated with the signal.

While there is apparently no proportionality factor in the expression for the vari-ance of Poisson noise, we can artificially introduce such a factor by scaling our exact data. Specifically, if we let ¯b_i^exact= η b_i^exactfor some positive η and define the Poisson data ¯b_i=P(¯bi^exact) =P(η b^exacti ), then clearlyE(¯bi) =V(¯bi) = η b^exact_i . Hence, if we are willing to accept noninteger data, then we can rescale ¯b_i^exact with η⁻¹ to obtain the data

bi ∼ P(η b^exacti )/η, i = 1, . . . , m,

which has meanE(bi) = b_i^exact and varianceV(bi) = η b^exact_i /η²= b^exact_i /η. We can use this mechanism to generate noninteger Poisson-like data in which we can control the proportionality factor between the exact data and the noise.

1We recall that when b_i^exactis large, thenP(b^exact_i )≈ N (b_i^exact, b^exact_i ).

Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

46 Chapter 3. Getting to Business: Discretizations of Linear Inverse Problems

3.6.3 Broad-Band Colored Noise

We conclude this section with a brief discussion of noise that is neither white nor

“white-noise–like.” Of special interest is noise that is still broad-band (i.e., all fre-quencies are represented) but dominated by either low-frequency (LF) components or high-frequency (HF) components. Both types of colored noise arise in applications, where they are usually referred to as broad-band colored noise; for brevity we shall refer to them as “LF noise” and “HF noise,” respectively.

There are many ways to generate colored noise from a white-noise signal, and we describe just one simple method here. Define the symmetric tridiagonal matrices

ΨLP= tridiag(1, 2, 1), ΨHF= tridiag(−1, 2, −1). (3.23) For example, for n = 4 we have

ΨLF=

⎛

⎜⎝

2 1 0 0

1 2 1 0

0 1 2 1

0 0 1 2

⎞

⎟⎠ , ΨHF=

⎛

⎜⎝

2 −1 0 0

−1 2 −1 0

0 −1 2 −1

0 0 −1 −2

⎞

⎟⎠ .

Then we can generate colored noise by multiplying these matrices to a white-noise signal:

eLF= ΨLFe, eHF= ΨHFe (3.24)

(in the language of signal processing, we apply a low-pass filter and, respectively, a high-pass filter to a white-noise signal to generate a colored-noise signal).

To study the covariance matrices for the colored-noise vectors, we need the following result for the eigenvalue decompositions of the two matrices Ψ_LF and Ψ_HF (which follows directly from Example 7.4 in [22], which gives closed-form expressions for the eigenvalues and vectors of tridiagonal matrices):

Ψ_LF= S diag(d₁, . . . , d_m) S^T, Ψ_HF= S diag(d_m, . . . , d₁) S^T, in which the diagonal elements 0 < dj < 4 are given by

dj = 2 + 2 cos!

πj /(m + 1)"

, j = 1, . . . , m, and S is an orthogonal matrix with elements

si j =

2/(m + 1) sin!

πi j /(m + 1)"

, i , j = 1, . . . , m.

The columns of S are samples of sine functions with increasing frequency, and hence they provide a spectral basis very similar to the Fourier basis. We conclude that the covariance matrices for the colored-noise vectors are given by

Cov(eLF) = η²S diag(d₁², . . . , d_m²) S^T, (3.25) Cov(e_HF) = η²S diag(d_m², . . . , d₁²) S^T, (3.26) where η is the standard deviation of the white noise in e in (3.24). Figure 3.13 shows plots of the diagonal elements in these expressions, and we clearly see that the

Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php