Países en desarrollo vecinos

We start with statements about the application of the Gauss–Newton method to the generally examined class of overdetermined residual functions R(x) whose Jacobian J (x) at the solution

x∗has a full rank. Subsequently, we state results for an extended class of residual functions.

Brown and Dennis show in [15] that the Gauss–Newton method converges q–linearly when the residual vector R : IRn → IRm _{in the nonlinear least–squares problem is overdetermined, i.e.}

m ≥ n, and the Jacobian J (x) of R(x) at the solution x∗ has a full column rank. If additionally,

the residual vector R(x) is zero at the solution x∗, then this method converges even q–quadratically

(see Theorem 1 – 2, Corollary 1 – 2 and Remark 2 in [15] as well as Dennis and Schnabel, Theorem 10.2.1 and Corollary 10.2.2 in [25]). The discussion by Dennis and Schnabel in [25] makes clear that the nonlinearity of the residual function and the size of the residual at the solution are important quantities in the proof of these results. They remark that the speed of convergence depends on these quantities in connection with the smallest eigenvalue of J (x∗)TJ (x∗). Moreover, if either

the relative nonlinearity or the relative residual size of the problem is too large, the Gauss–Newton method may not converge at all.

The convergence results of Brown and Dennis have the disadvantage that they do not consider the invariance of the Gauss–Newton sequence under unitary transformations (see e.g. Subsection 3.2.3). This aspect is considered by Häußler in [50]. He shows that for overdetermined residual functions with rank(J (x∗)) = n the Gauss–Newton method converges q–linearly or q–quadratically,

respectively, under assumptions which are also invariant under unitary transformations (see Theorems 2.1, 2.5, and 2.7 in [50]).

Deuflhard and Apostolescu examine in [29] the Gauss–Newton method also for overdetermined residual vectors; however, they drop the assumption about the full rank of the Jacobian at the solution. They show under unitary invariant assumptions that the Gauss–Newton method converges locally for adequate nonlinear least–squares problems if the norm minimal solution of the Gauss– Newton subproblems is chosen for the correction of the Gauss–Newton iterates (see Theorem 1 in [29]). On this occasion, they call a nonlinear least–squares problem adequate, if and only if, there exists some convex neighborhood of the solution x∗so that

3.2 Local Convergence Analysis of the Inexact Gauss–Newton Method 33

holds for all elements x, y in this neighborhood. A combination of the critical point condition of x∗,

i.e. J (x∗)TR(x∗) = J (x∗)+R(x∗) = 0, with the condition (3.2.1) yields

kJ (y)+R(x∗)k2 ≤ χ(x∗) ky − x∗k2

such that condition (3.2.1) can be interpreted as some kind of “small residual” condition on the residual vector R(x) [29]. The result of Deuflhard and Apostolescu does not contain any statements about the rate of convergence of kxk− x∗k2, where xkis the kth Gauss–Newton iterate.

The local convergence analyses of the Gauss–Newton method by Boggs [11], Deuflhard and Heindl [30], and Bock [10] do not distinguish between overdetermined and underdetermined residual vectors R(x). Boggs proves in [11] the local convergence of the Gauss–Newton method if the norm minimal solution of the Gauss–Newton subproblems is chosen for the update of the Gauss–Newton iterates (see Theorem 3.2 in connection with his statements about the bounds on the step size in [11]). This result is valid for residual functions whose Jacobian J (x) has a constant rank and J (x)TR(x) is Lipschitz continuous in a convex neighborhood of the solution x∗and if the

largest eigenvalue of J (x∗)TJ (x∗) is smaller than 2. If this last assumption on the largest eigenvalue

of J (x∗)TJ (x∗) is not satisfied, then he obtains the local convergence for a damped Gauss–Newton

method.

Deuflhard and Heindl present in [30] an unitary invariant convergence theorem for a class of generalized Gauss–Newton methods. They prove under assumptions that are invariant under unitary transformations the local convergence to a critical point x∗ for this class of generalized Gauss–

Newton methods (see Theorem 4 in [30]). The ordinary Gauss–Newton method where the Gauss– Newton iterates are updated by using the norm–minimal solution of the Gauss–Newton subproblems is contained in this class. The critical condition in their result is again the condition (3.2.1).

Furthermore, Bock proves in [10] the r–linear convergence of the Gauss–Newton sequence if the norm minimal solution of the Gauss–Newton subproblems is chosen for the correction of the Gauss–Newton iterate (see Theorem 3.1.44 in [10]). He introduces a generalized inverse which makes in his case an uniform treatment of constrained as well as unconstrained nonlinear least– squares problems possible. The critical condition in his result is given by

 J (y)(+)− J (x)(+) Q(x) 2 ≤ χ(x) ky − xk₂ with χ(x) ≤ χ < 1

where J (x)(+)is a generalized inverse of the Jacobian J (x) and Q(x) is given by

Q(x) := R(x) − J (x) J (x)(+)R(x) = I − J (x) J (x)(+) R(x) .

If unconstrained nonlinear least–squares problems are treated, the generalized inverse J (x)(+) of the Jacobian J (x) in Bock’s examination is just the Moore–Penrose inverse J (x)+ so that in this case the above given condition is identical to condition (3.2.1) by Deuflhard and Heindl because of the identity J (x)+(I − J (x) J (x)+) = 0.

Moreover, Dennis and Steihaug examined in [26] the convergence rate of an inexact version of the Gauss–Newton method in the overdetermined, full rank case. Under similar assumptions as Dennis and Schnabel, they showed that the inexact Gauss–Newton iterates converge at least

34 3 Convergence

q–linearly to the solution in a weighted norm (see Theorem 3.1 in [26]). The convergence of this

sequence, again, depends on the nonlinearity and the size of the residual of the problem. Additionally, since the linear Gauss–Newton subproblems are only solved inexactly, the residuals of the linearized problems, i.e. J (xi)Tri= J (xi)T(J (xi) δxi+ R(xi)) , influence the convergence of

the iterates to a solution.

3.2.2 Difficulties in the Convergence Analysis of the Inexact Gauss–Newton Method