III. DERECHOS DE LOS OKUPADORES
2.2 La protección de la posesión: la protección de los okupas
2.2.3 Los interdictos para recuperar la posesión
This short appendix summarizes some of the early work on the method which is com-monly referred to as “Tikhonov regularization” today. As we shall see, this name is not quite fair to several authors of these early papers, especially Phillips and Twomey (who are occasionally mentioned along with Tikhonov).
Perhaps the first author to describe a scheme that is equivalent to Tikhonov regu-larization was James Riley, who, in his paper [60] from 1955, considered ill-conditioned systems A x = b with a symmetric positive (semi)definite coefficient matrix. He pro-posed to solve instead the system (A+α I) x = b, where α is a small positive constant.
In the same paper, Riley also suggested an iterative scheme which is now known as iterated Tikhonov regularization; cf. Section 5.1.5 in [32].
The first paper devoted specifically to inverse problems was published by D. L.
Phillips [58] in 1962. In this paper A is a square matrix obtained from discretization of a first-kind Fredholm integral equation by means of a quadrature rule, and L is the tridiagonal matrix Lz2. Phillips arrived at the formulation in (8.2), but without matrix notation, and then proposed to compute the regularized solution as xλ = (A + λ2A−TLTL)−1b, using our notation. It is not clear whether Phillips computed A−1 explicitly, and the paper does not recognize (8.2) as a least squares problem.
In his 1963 paper [69], S. Twomey reformulated Phillips’s expression for xλvia the least squares formulation and obtained the well-known “regularized normal equations”
expression for the solution:
xL,λ=!
ATA + λ2LTL"−1 ATb,
still with L = Lz2. He also proposed to include an a priori estimate x0, but only in con-nection with the choice L = I, leading to the formula xλ=!
ATA + λ2I"−1!
ATb + λ2x0
"
. A. N. Tikhonov’s paper [65] from 1963 is formulated in a much more general setting: he considered the problem K f = g, where f and g are functions and K is an integral operator. Tikhonov proposed the formulation (8.1) with the particular smoothing norm
S(f ) = b
a
!v (s) f (s)2+ w (s) f(s)2"
Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php d s,
204 Appendix C. Early Work on “Tikhonov Regularization”
where v and w are positive weight functions. Turning to computations, Tikhonov used the midpoint quadrature rule to arrive at the problem
minx
1A x − b22+ λ2
D1/2v x22+Dw1/2L x22
2 ,
in which Dv and Dw are diagonal weight matrices corresponding to v and w , and L = bidiag(−1, 1). Via the “regularized normal equations” he then derived the expression xL,λ=!
ATA + λ2(Dv + LTDwL)"−1 ATb.
In 1965 Gene H. Golub [20] was the first to propose a modern approach to solving (8.2) via the least squares formulation (8.3) and QR factorization of the associated coefficient matrix. Golub proposed this approach in connection with Riley’s iterative scheme, which includes the computation of xλ as the first step. G. Ribiere [59] also proposed the QR-based approach to computing xλin 1967.
Stochastic perspectives on Tikhonov regularization go back to the paper [16] by M. Foster from 1961, which predates the Phillips and Tikhonov papers. This paper introduces the “regularized normal equations” formulation of general-form Tikhonov regularization, including the prewhitening from Section 4.8.2. In this setting, the matrix λ2LTL (in our notation) represents the inverse of the covariance matrix for the solution.
In 1966, V. F. Turchin [68] introduced a statistical ensemble of smooth functions that satisfy the Fredholm integral equation (2.2) within data errors. Joel Franklin’s paper [17] from 1970 revisits the formulation in general form in the setting of random processes over Hilbert spaces.
In much of the statistics literature, Tikhonov regularization is known as ridge regression, which seems to date back to the papers [44], [45] from 1970 by Hoerl and Kennard. The same year, Marquardt [52] used this setting as the basis for an analysis of his iterative algorithm from 1963 for solving nonlinear least squares problems [53], and which incorporates standard-form Tikhonov regularization in each step.
Finally, it should be mentioned that Franklin [18] in 1978, in connection with symmetric positive (semi)definite matrices A and B, proposed the regularized solution
¯
xλ= (A + α B)−1b, where α is a positive scalar, which nicely connects back to Riley’s 1955 paper.
Downloaded 07/26/14 to 129.107.136.153. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
Problems
scientific computing. The intended audiences are researchers and practitioners using the methods and upper level undergraduates in mathematics, engineering, and computational science.
Books in this series not only provide the mathematical background for a method or class of methods used in solving a specific problem but also explain how the method can be developed into an algorithm and translated into software. The books describe the range of applicability of a method and give guidance on troubleshooting solvers and interpreting results. The theory is presented at a level accessible to the practitioner. MATLAB® software is the preferred language for codes presented since it can be used across a wide variety of platforms and is an excellent environment for prototyping, testing, and problem solving.
The series is intended to provide guides to numerical algorithms that are readily accessible, contain practical advice not easily found elsewhere, and include understandable codes that implement the algorithms.
Editorial Board
Series Volumes
Hansen, P. C., Discrete Inverse Problems: Insight and Algorithms Modersitzki, J., FAIR: Flexible Algorithms for Image Registration
Chan, R. H.-F. and Jin, X.-Q., An Introduction to Iterative Toeplitz Solvers Eldén, L., Matrix Methods in Data Mining and Pattern Recognition
Hansen, P. C., Nagy, J. G., and O’Leary, D. P., Deblurring Images: Matrices, Spectra, and Filtering Davis, T. A., Direct Methods for Sparse Linear Systems
Kelley, C. T., Solving Nonlinear Equations with Newton’s Method Uri M. Ascher
University of British Columbia Howard Elman
University of Maryland Mark Embree
Rice University Michael T. Heath
University of Illinois at Urbana-Champaign C. T. Kelley
North Carolina State University Beatrice Meini
University of Pisa
Cleve Moler The MathWorks, Inc.
James G. Nagy Emory University Dianne P. O’Leary University of Maryland Danny Sorensen Rice University Henry Wolkowicz University of Waterloo