Let(X,k · kX),(Y,k · kY)be reflexive, separable Banach spaces We will also assume that the
data points,Ψn={ξi}ni=1 ⊂Xfori= 1,2, . . . , nare iid random elements with common law
P. As beforeµ = (µ1, µ2, . . . , µk) but now the cluster centersµj ∈ Y for eachj. The cost
function isd:X×Y →[0,∞).
The energy functions associated with thek-means algorithm in this setting are slightly different to those used previously:
gµ:X→R, gµ(x) = k ^ j=1 d(x, µj), fn(ω):Yk→R, fn(ω)(µ) =Pn(ω)gµ+λr(µ), (3.13) f∞:Yk→R, f∞(µ) =P gµ+λr(µ). (3.14)
The aim of this section is to show the convergence result:
ˆ
θn(ω)= inf
µ∈Ykf
(ω)
n (µ)→ inf
µ∈Ykf∞(µ) =θ and asn→ ∞forP-almost everyω and that minimizers converge (almost surely).
The key assumptions are given in Assumption 2; they imply thatfn(ω)is weakly lower
semi-continuous and coercive. In particular, Assumption 2.2 allows us to prove the lim inf in- equality as we did for Theorem 3.2.2. Assumption 2.1 is likely to mean that our convergence results are limited to the case of bounded noise. In fact, when applying the problem to the smoothing-data association problem, it is necessary to bound the noise in order for Assump- tion 2.5 to hold. Assumption 2.5 implies thatfn(ω)is (uniformly) coercive and hence allows us
to easily bound the set of minimizers. In the next chapter we will remove the bounded noise assumption for the smoothing-data association problem. Assumption 2.3 is a measurability con- dition we require in order to integrate and the weak lower semi-continuity ofris needed for the to obtain the lim inf inequality in theΓ-convergence proof.
We note that, sinceP d(·, µ1)≤supx∈supp(P)d(x, µ1)<∞, we havef∞(µ)<∞for
everyµ∈Yk(and sincer(µ)<∞for eachµ∈Yk).
Assumptions 2. We have the following assumptions ond:X×Y →[0,∞),r :Yk→[0,∞)
2.1. For ally ∈Y we havesupx∈supp(P)d(x, y)<∞wheresupp(P)⊆Xis the support of
P.
2.2. For eachx∈Xandy ∈Y we have that ifxm→xandyn* yasn, m→ ∞then
lim inf
n,m→∞d(xm, yn)≥d(x, y) and mlim→∞d(xm, y) =d(x, y).
2.3. For everyy∈Y we have thatd(·, y)isX-measurable. 2.4. ris weakly lower semi-continuous.
2.5. ris coercive.
We will follow the structure of Section 3.2. We start by showing that under the above conditionsfn(ω)Γ-converges tof∞. We then show that the regularization term guarantees that
the minimizers tofn(ω)lie in a bounded set. An application of Theorem 2.2.1 gives the desired
convergence result. Since we were able to restrict our analysis to a weakly compact subset ofY
we are easily able to deduce the existence of a weakly convergent subsequence.
Similarly to the previous section on the product spaceYk we use the analogous norm
kµkk:= maxjkµjkY.
Theorem 3.3.4. Let (X,k · kX) and (Y,k · kY) be separable and reflexive Banach spaces. Assumer : Yk → [0,∞), d : X ×Y → [0,∞) and the probability measureP on (X,X)
satisfy the conditions in Assumptions 2. For independent samples{ξiω)}n
i=1fromP defineP (ω)
n
to be the empirical measure and fn(ω) : Yk → R andf∞ : Yk → R by (3.13) and(3.14)
respectively and whereλ >0. Then
f∞= Γ-lim
n f
(ω)
n
forP-almost everyω.
Proof. Define
Ω0 =nω ∈Ω :Pn(ω)⇒Po∩nω∈Ω :ξi(ω)∈supp(P)∀i∈N o
.
ThenP(Ω0) = 1. For the remainder of the proof we consider an arbitraryω ∈Ω0. We start with
the lim inf inequality. Letµ(n)* µthen lim inf
n→∞ f (ω)
n (µ(n))≥f∞(µ)
follows (as in the proof of Theorem 3.2.2) by applying Theorem 1.1 in [64] and the fact thatr
is weakly lower semi-continuous.
We now establish the existence of a recovery sequence. Letµ ∈Yk and letµ(n) =µ. We want to show
lim
n→∞f (ω)
Clearly this is equivalent to showing that
lim
n→∞P (ω)
n gµ=P gµ.
Nowgµare continuous by assumption ond. LetM = supx∈supp(P)d(x, µ1)<∞and note that
gµ(x)≤M for allx∈supp(P)and therefore bounded. HencePn(ω)gµ→P gµ.
Proposition 3.3.5. Assuming the conditions of Theorem 3.3.4, then forP-almost everyωthere
existsN <∞andR >0such that
min µ∈Ykf (ω) n (µ) = min kµkk≤R fn(ω)(µ)< inf kµkk>R fn(ω)(µ) ∀n≥N. In particularRis independent ofn. Proof. Let Ω00 = n ω∈Ω0 :Pn(ω) ⇒P o ∩nω ∈Ω0 :Pn(ω)d(·,0)→P d(·,0) o .
Then, for every ω ∈ Ω00, fn(ω)(0) → f∞(0) < ∞ where with a slight abuse of notation we
denote the zero element in bothY andYkby0. TakeN sufficiently large so that
fn(ω)(0)≤f∞(0) + 1 for alln≥N.
Thenminµ∈Ykf
(ω)
n (µ) ≤ f∞(0) + 1for alln ≥ N. By coercivity of r there existsR such
that ifkµkk > Rthen λr(µ) ≥ f∞(0) + 1. Therefore any suchµis not a minimizer and in
particular any minimizer must be contained in the setµ∈Yk:kµkk≤R .
The convergence results now follows by applying Theorem 3.3.4 and Proposition 3.3.5 to Theorem 2.2.1.
Theorem 3.3.6. Assuming the conditions of Theorem 3.3.4 and Proposition 3.3.5 the minimiza- tion problem associated with thek-means method converges in the following sense:
min
µ∈Ykf∞(µ) = limn→∞µmin∈Ykf
(ω)
n (µ)
forP-almost every ω. Furthermore any sequence of minimizersµ(n) offn(ω) is almost surely
weakly precompact and any weak limit point minimizesf∞.
It was not necessary to assume that cluster centers are in a common space. A trivial generalization would allow eachµj ∈Y(j)with the cost and regularization terms appropriately
defined; in this setting Theorem 3.3.6 holds.