La función social de la propiedad

The presentation so far has focused on the 2-norm as a means for controlling (or suppressing) the error in the regularized solution. The normx2 enters explicitly in the Tikhonov formulation (4.8), and it is also part of the alternative formulation (4.5) of the TSVD method.

While the normx2is indeed useful in a number of applications, it is not always the best choice. In order to introduce a more general formulation, let us return to the continuous formulation of the first-kind Fredholm integral equation from Chapter 2.

In this setting, the residual norm for the generic problem in (2.2) takes the form

R(f ) =

 1 0

K(s, t) f (t) d t− g(s)

2

Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php .

172 Chapter 8. Beyond the 2-Norm: The Use of Discrete Smoothing Norms In the same setting, we can introduce a smoothing norm S(f ) that measures the regularity of the solution f . Common choices of S(f ) belong to the family given by

S(f ) =f^{(d )}2=

 1 0

!f^{(d )}(t)"2

d t

1/2

, d = 0, 1, 2, . . . ,

where f^{(d )}denotes the d th derivative of f . Then we can write the Tikhonov regular-ization problem for f in the form

min%

R(f )²+ λ²S(f )²&

, (8.1)

where λ plays the same role as in the discrete setting. Clearly, the Tikhonov for-mulation in (4.8) is merely a discrete version of this general Tikhonov problem with S(f ) =f 2.

Returning to the discrete setting, we can also formulate a general version of the Tikhonov problem by replacing the normx2 with a discretization of the smooth-ing norm S(f ). The general form of the discrete smoothsmooth-ing norm takes the form

L x2, where L is a discrete approximation of a derivative operator, and the associ-ated Tikhonov regularization problem in general form thus takes the form

minx

%A x − b²2+ λ²L x²2

&

. (8.2)

The matrix L is p× n with no restrictions on the dimension p.

If L is invertible, such that L⁻¹ exists, then the solution to (8.2) can be written as x_L,λ= L⁻¹x¯_λ, where ¯x_λsolves the standard-form Tikhonov problem

minx¯ {(A L⁻¹) ¯x− b²2+ λ²¯x²2}.

The multiplication with L⁻¹ in the back-transformation xλ = L⁻¹x¯λ represents inte-gration, which yields additional smoothness in the Tikhonov solution, compared to the choice L = I. The same is also true for more general rectangular and noninvertible smoothing matrices L.

Similar to the standard-form problem obtained for L = I, the general-form Tikhonov solution xL,λ is computed by recognizing that (8.2) is a linear least squares problem of the form

minx

 A λ L

 x−

b 0



2

. (8.3)

The general-form Tikhonov solution xL,λis unique when the coefficient matrix in (8.3) has full rank, i.e., when the null spaces of A and L intersect trivially:

N (A) ∩ N (L) = ∅.

Since multiplication with A represents a smoothing operation, it is unlikely that a smooth null vector of L (if L is rank-deficient) is also a null vector of A.

Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

8.1. Tikhonov Regularization in General Form 173 Various choices of the matrix L are discussed in the next section; two common choices of L are the rectangular matrices

L₁=

⎛

⎝−1 1 . .. . ..

−1 1

⎞

⎠ ∈ R^(n−1)×n, (8.4)

L2=

⎛

⎝

1 −2 1

. .. . .. . ..

1 −2 1

⎞

⎠ ∈ R^(n−2)×n, (8.5)

which represent the first and second derivative operators, respectively (in Regulariza-tion Tools use get_l(n,1) and get_l(n,2) to compute these matrices). Thus, the discrete smoothing normL x2, with L given by I, L1, or L2, represents the contin-uous smoothing norms S(f ) = f 2, f^2, and f^2, respectively. We recall the underlying assumption that the solution f is represented on a regular grid, and that a scaling factor (related to the grid size) is absorbed in the parameter λ.

To illustrate the improved performance of the general-form formulation, consider a simple ill-posed problem with missing data. Specifically, let x be given as samples of a function, and let the right-hand side be given by a subset of these samples, e.g.,

b = A x , A =

I_left 0 0 0 0 I_right

 ,

where Ileft and Iright are two identity matrices. Figure 8.1 shows the solution x (con-sisting of samples of the sine function), as well as three reconstructions obtained with the three discrete smoothing normsx2,L1x2, andL2x2. For this problem, the solution is independent of λ. Clearly, the first choice is bad: the missing data are set to zero in order to minimize the 2-norm of the solution. The choiceL1x2 produces a linear interpolation in the interval with missing data, while the choiceL2x2produces a quadratic interpolation here.

In certain applications it can be advantageous to use a smoothing norm S(f ) that incorporates a combination of several derivatives of the function f , namely, the weighted Sobolev norm defined by

S(f ) =



α₀f ²2+ α₁f^²2+· · · + αpf^{(d )}²2

1/2

(the standard Sobolev norm corresponds to α0 = · · · = αd = 1). Such smooth-ing norms are particularly handy in 2D and 3D problems where we need to en-force different regularity on the solution in different directions, e.g., by choosing S(f ) = !

∂f /∂x²2+∂²f /∂y²²2

"1/2

in a 2D problem. There is a vast number of choices of combinations, and the best choice is always problem dependent.

In the discrete setting, weighted Sobolev norms are treated by “stacking” the required matrices that represent the derivative operators. For example, if we use the two matrices in (8.4)–(8.5), then the matrix L that corresponds to the Sobolev norm

Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php

174 Chapter 8. Beyond the 2-Norm: The Use of Discrete Smoothing Norms

0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

The model

0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

Reconstr., L = I

0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

Reconstr., L = L 1

0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

Reconstr., L = L 2

Figure 8.1. Illustration of the missing data problem. We show Tikhonov solutions for three choices of the discrete smoothing normL x2.

with d = 2 takes the form

L =

⎛

⎝ I L1

L2

⎞

⎠ ∈ R^(3n−3)×n.

If we compute the QR factorization of this “tall” matrix, i.e., L = Q R, then we can replace L with the square, banded, and upper triangular matrix R. To see that this is valid, just notice that

L x²2= (L x )^T(L x ) = (Q R x )^T(Q R x ) = x^TR^TQ^TQ R x = x^TR^TR x =R x²2

(since the matrix Q has orthonormal columns). Similar techniques may apply to problems in two and three dimensions. For large-scale problems, such factorizations of L may be useful, provided that the triangular factor R remains sparse.

We finish this brief discussion of general-form regularization methods with a discussion of the null space of L. It is often convenient to allow the matrix L to have a nontrivial null space

N (L) ≡ {x ∈ Rⁿ: L x = 0}.

Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php