The first two results show that the filter or PLPF envelope converges to zero, if the filter or PLPF is augmented infinitely many times.
Lemma 4.15. Let the non-monotone filter Fmag,k lf be augmented infinitely many times,
f xk
k be bounded below and
θ xk k be bounded above. Then, the sequence {δk}k con- verges to zero.
Proof. The statement is proven by contradiction and follows ideas of Nocedal et al. [153, Theorem 5.1] and Shen et al. [177, Lemma 3.1]. Assume there exists an infinite index set K such that the filter Fmag,k(lf) is augmented (by assumption of this lemma) and δk≥ ϵ > 0 for
k ∈ K .
Now assume thatf xk
K is not bounded above. Then, there exists an index set K′⊆ K such that the sequencef xk
K′ is monotonically increasing. Hence, since γf> 0 it follows f xk+1+ γfδk> f xk
,
for all k ∈ K′, i.e., all the iterates xk+1, k ∈ K′are dominated with respect to the objective function. This is why, there must be one of the xk, xk−1, . . . , xk−lfiterates such that xk+1is not
dominated with respect to the constraint violation. Otherwise, xk+1would not be acceptable
to the non-monotone filter and therefore k /∈ K . Thus, θ xk+1+ γfδk≤ max
i=0,...,lfθ x
(k−i)+.
By the same argument, this can be extended to the next lfiterations, leading to max i=1,...,lf+1θ x k+i≤ max i=0,...,lfθ x (k−i)+− γfδk≤ max i=0,...,lfθ x (k−i)+− ϵ
4.3. Global Convergence Analysis 93
and thus max
i=1,...,lf+1θ x
k+i→ −∞
for k ∈ K′→ ∞, which is a contradiction to θ(x) = ∥g(x)∥
2≥ 0. This implies that
f xk K is bounded above.
Because the sequence f xk , θ xk K is bounded, it exists an index set K′′⊆ K and ¯f, ¯θsuch that f xk , θ xk → ¯f, ¯θ for k ∈ K′′→ ∞. For k ∈ K′′large enough, it follows that f xk+1−j , θ xk+1−j ∈ B1 2ϵ ¯f, ¯θ for j = 0, . . . , lf+ 1. Therefore, θ xk+1− θ xk+1−j ≤ θ xk+1− ¯θ + θ xk+1−j− ¯θ < ϵ, f xk+1− f xk+1−j ≤ f xk+1− ¯f + f xk+1−j− ¯f < ϵ and thus, θ xk+1> θ xk+1−j− ϵ ≥ θ xk+1−j− δk, f xk+1> f xk+1−j− ϵ ≥ f xk+1−j− δ k
for j = 1, . . . , lf+ 1 and k ∈ K′′, i.e., xk+1 is dominated by the lf+ 1 points xk+1−j, j = 1, . . . , lf+ 1. However, since k ∈ K , the point xk+1 is acceptable to the non-monotone filter – a contradiction. All together, this implies that {δk}k converges to zero.
Lemma 4.16. Let the non-monotone PLPF Fmag,k lf be augmented infinitely many times,
f xk
k be bounded below and
θ xk k be bounded above. Then, the sequence {δk}k con- verges to zero.
Proof. Let D Fmag,k(lf)
be the acceptable region of the PLPF and D′ F
mag,k(lf)
the one of a filter with the same points Fmag,k(lf) and envelope δk for every k. In the following it will be
shown, that if an iterate is acceptable to the PLPF, it will also be for the filter defined above and thus, that filter would be augmented with every PLPF augmentation.
Let K be an infinite index set of iterations in which the PLPF is augmented. Assume that there exists an index k ∈ K such that the non-monotone filter defined above would not be augmented. Then, there exists an index set K′⊆ {0, . . . , k} with K′ = l
f+ 1 such that f xk+1+ γ fδk> f xj , and θ xk+1+ γfδk> θ xj
for all j ∈ K′. This implies
f xk+1+ τθ xk+1+ (1 + τ) γfδk> f xj
for all j ∈ K′and τ ≥ 0. Thus, f xk+1, θ xk+1is dominated by l
f+1 elements with respect to the PLPF condition, i.e.,
f xk+1, θ xk+1∈ D F/ mag,k(lf)
.
This is a contradiction to the assumption that the PLPF criterion was accepted for all k ∈ K . It follows, that the filter is augmented infinitely many times. Then, Lemma 4.15 implies that {δk}kconverges to zero.
Since the filter or PLPF envelope δkfor the magic step is chosen to measure the KKT error of
the problem (NLP+), the convergence of δk to zero directly leads to the following outcome
for infinitely many performed magic steps.
Lemma 4.17. Suppose the Assumptions 4.12 hold. If the magic step is executed infinitely many
times in Step L-6, there exists an index set K such that the sequence xk, yk/π
k−, zk/πk−
K converges to a first-order optimal solution of (NLP+).
Proof. Let K be an index set of iterations in which the magic step is accepted, i.e., the filter or PLPF Fmag,k(lf) is augmented, and for which xk→ x∗holds. The latter exists due to Assump- tion 4.12 (ii). Together with Assumption 4.12 (i) it follows that f xk K and θ xk K are bounded. Then, by Lemma 4.15 or Lemma 4.16 it follows that
δk= ∇f xk+π1k − ∇g xkyk− 1 πk−z k 2 2 + g xk 22+ π1k − Xkzk 2 2 → 0 for k ∈ K → ∞.