By Lemma 3.33 this is bounded by X
b∈X
µ(b)P(τa,b > n) ≤ sup
a,b P(τ
a,b > n) ≤ (1 − β(Q))n. (3.42)
by Lemma 3.18. This finishes the proof of (3.39).
Let µ and ν be two invariant measures for Q. As we did in Lemma 3.33, we construct two chains Xnµ and Xnν with initial states randomly chosen according with µ and ν, respectively. Then,
|ν(y) − µ(y)| = X x ν(x)P(Xnx = y) − X z µ(z)P(Xnz = y) = X x X z ν(x)µ(z) (P(Xnx= y) − P(Xnz = y)) ≤ X x X z ν(x)µ(z) sup a,b P(τ a,b> n) ≤ (1 − β(Q))n, (3.43)
using Lemma 3.33 in the first inequality and (3.19) in the second. Since the bound (3.43) holds for all n and β(Q) > 0 by hypothesis, µ = ν. This shows uniqueness.
The assumption β(Q) > 0 is of course very restrictive. The following corollary allows us to get a result as Theorem 3.38 in a more general case.
3.3. PERIODIC AND APERIODIC CHAINS 41 Corollary 3.44 Let Q be a transition matrix on a countable state space X satisfying β(Qk) > 0 for some k ≥ 1. Then the chain has a unique invariant measure µ and sup a,y |P(X a n = y) − µ(y)| ≤ (1 − β(Q k))n/k. (3.45)
Proof. Left to the reader.
A natural question arises: which are the transition matrices on a count- able state space X having β(Qj) strictly positive for some j ≥ 1. One example for the infinite countable case is the house-of-cards process of Ex- ample 1.45. If for this process there exists an ε > 0 such that ak < 1 − ε for
all k, then β(Q) > ε. We do not discuss further the infinite countable case. A related question in the finite case is the following: which are the tran- sition matrices on a finite state space X having all entries positive starting from some power? The examples of irreducible matrices proposed in the pre- vious section show that the irreducibility condition is necessary. However, as it is shown in the next section, it is not sufficient.
3.3
Periodic and aperiodic chains
Let us start with an example.
Example 3.46 Let X = {1, 2} and the transition matrix Q be given by Q = 0 1
1 0
It is clear that this matrix corresponds to an irreducible process. However, any power has null entries. Indeed, for all k ≥ 0, we have
Q2k = 1 0 0 1 and Q2k+1= 0 1 1 0
42 CHAPTER 3. CONVERGENCE AND LOSS OF MEMORY The problem of this matrix is that the transitions from 1 to 2 or from 2 to 1 can only occur in an odd number of steps, while the transitions from 1 to 1, or from 2 to 2, are only possible in an even number of steps. Anyway this matrix accepts a unique invariant measure, the uniform measure in X . This type of situation motivates the notion of periodicity which we introduce in the next definition.
Definition 3.47 Assume Q to be the transition matrix of a Markov chain on X . An element x of X is called periodic of period d, if
mcd {n ≥ 1 : Qn(x, y) > 0} = d. The element will be called aperiodic if d = 1.
For example, in the matrix of the Example 3.46 both states 1 and 2 are periodic of period 2.
We omit the proof of next two propositions. They are elementary and of purely algebraic character and can be found in introductory books of Markov chains. The first one says that for irreducible Markov chains the period is a solidarity property, that is, all states have the same period.
Proposition 3.48 Let Q be a transition matrix on X . If Q is irreducible, then all states of X have the same period.
The proposition allows us to call irreducible matrices of chain of period d or aperiodic chain
Proposition 3.49 Let Q be an irreducible transition matrix on a finite set X . If Q is irreducible and aperiodic, then there exists an integer k such that Qj has all entries positive for all j ≥ k.
Proof. Omitted. It can be found in H¨aggstr¨om (2000), Theorem 4.1, for instance.
Irreducible periodic matrices induce a partition of the state space in classes of equivalence. Let Q be a matrix with period d on X . We say that x
3.4. DOBRUSHIN’S ERGODICITY COEFFICIENT 43 is equivalent to y if there exists a positive integer k such that Qkd(x, y) > 0.
Then X = X1, . . . , Xd, where Xi contains equivalent states and are called
equivalent classes.
Proposition 3.50 Let Q be a irreducible matrix with period d. Then Qd is
aperiodic in each one of the classes of equivalence X1, . . . , Xd. Let µ1, . . . , µd
be invariant measures for Qd on X1, . . . , Xd, respectively. Then the measure
µ defined by µ(x) := 1 d X i µi(x) (3.51)
is an invariant measure for Q.
Proof. It is left as an exercise for the reader.
The last result of this section says that the positivity of all elements of a power of an irreducible matrix Q implies the positivity of β(Q).
Lemma 3.52 Let Q be a transition matrix on a finite set X . If there exists an integer k such that Qj has all entries positive for all j ≥ k then β(Qj) > 0.
Proof. It is clear that Qj(x, y) > 0 for all x, y ∈ X implies β(Qj) > 0.
3.4
Dobrushin’s ergodicity coefficient
We present another coupling to obtain a better speed of loss of memory of the chain. Let X be a finite or countable state space.
Definition 3.53 The Dobrushin’s ergodicity coefficient of a transition ma- trix Q on X is defined by α(Q) = inf a,b X x∈X min{Q(a, x), Q(b, x)}.
44 CHAPTER 3. CONVERGENCE AND LOSS OF MEMORY Theorem 3.54 If Q is a transition matrix on a countable state space X , then there exists a coupling (joint construction of the chains) (Xna, Xnb : n ∈ N) constructed with a function eF such that the meeting time of the coupling
e
F satisfies
P(τa,b> n) ≤ (1 − α(Q))n. (3.55) To prove this theorem we use the Dobrushin coupling.
Definition 3.56 (Dobrushin coupling) Let Q be a transition probability matrix on a countable state space X . We construct a family of partitions of [0, 1] as in Theorem 3.18. But now we double label each partition, in such a way that the common part of the families Ia(b, y) ∩ Ib(a, y) be as large as
possible.
We assume again that X = {1, 2, . . .}. For each fixed elements a and b of X define
Ja,b(y) := [la,b(y − 1), la,b(y)) (3.57) where
la,b(y) := 0, if y = 0;
la,b(y − 1) + min{Q(a, y), Q(b, y)} if y ≥ 1. (3.58)
Let la,b(∞) := lim
y→∞la,b(y).
Displays (3.57) and (3.58) define a partition of the interval [0, la,b(∞)]. We
need to partition the complementary interval (la,b(∞), 1]. Since the common parts have already been used, we need to fit the rests in such a way that the total lengths equal the transition probabilities. We give now an example of this construction. Define
e
Jb(a, y) = [˜lb(a, y − 1), ˜lb(a, y)) where
˜
lb(a, y) = l
a,b(∞), if y = 0;
˜
lb(a, y − 1) + max{0, (Q(a, y) − Q(b, y))}, if y ≥ 1.
Finally we define
3.4. DOBRUSHIN’S ERGODICITY COEFFICIENT 45 It is easy to see that for all b the following identity holds
|Ib(a, y)| = Q(a, y).
Define the function eF : X × X × [0, 1] → X × X as in Definition 3.1:
e F (a, b; u) = N X y=1 N X z=1
(y, z)1{u ∈ Ib(a, y) ∩ Ia(b, z)}. (3.59) In other words, eF (a, b; u) = (y, z), if and only if u ∈ Ib(a, y) ∩ Ia(b, z). Notice
that
y = z implies Ib(a, y) ∩ Ia(b, z) Hence, for any a and b,
if u < la,b(∞), then eF (a, b; u) = (x, x) for some x ∈ X . (3.60) We construct the coupling (Xa
n, Xnb)n≥0 as follows:
(Xna, Xnb) = (a, b), if n = 0 ; e
F (Xn−1a , Xn−1b ; Un), if n ≥ 1,
(3.61) where (U1, U2, · · ·) is a sequence of iid uniformly distributed in [0, 1]. The
process so defined will be called Dobrushin coupling.
Proof of Theorem 3.54. With Dobrushin coupling in hands, the rest of the proof follows those of Theorem 3.18 with the only difference that now the coincidence interval changes from step to step, as a function of the current state of the coupled chains. Let
e
τa,b = min{n ≥ 1 : Un < lX
a
n−1,Xn−1b (∞)} .
The law of τea,b is stochastically dominated by a geometric random variable:
P(eτa,b > n) = P(U1 > lX
a 0,X0b(∞), · · · , U n > lX a n−1,Xn−1b (∞)) ≤ P(U1 > inf x,yl x,y(∞), · · · , U n> inf x,yl x,y(∞)) = n Y i=1 P(Ui > inf x,yl x,y (∞)) = (1 − inf x,yl x,y (∞))n.
46 CHAPTER 3. CONVERGENCE AND LOSS OF MEMORY To conclude observe that
inf
x,yl
x,y(∞) = α(Q).
Finally we can state the convergence theorem.
Theorem 3.62 If Q is an irreducible aperiodic transition matrix on a count- able state space X , then
sup(a,b)|P(Xna = b) − µ(b)| ≤ (1 − α(Q
k))nk, (3.63)
where µ is the unique invariant probability for the chain and k is the smallest integer for which all elements of Qk are strictly positive.
Proof. Follows as in Theorem 3.38, substituting the bound (3.19) by the bound (3.55).