Kandori et al (1993) & Young (1993) use a result due to Friedlin & Wentzell (1984) to characterise the stationary distribution o f a Markov chain. This enables them to analyse the behaviour o f the distribution as the mutation rate becomes vanishingly small. Using this characterisation, Kandori et al show that in the limit the stationary distribution becomes concentrated on a set o f states which they call long-run equilibria, and that these states have the property that they require the low est number o f mutations to move to from all other states taken together. Young shows that to find the long-run equilibria, it is sufficient to look at the number o f mutations required to move between the set o f equilibria rather than the set o f states. In this section, we give a brief review o f the stochastic techniques developed in these papers.
2.1.1 Friedlin & Wentzell
Consider a finite Markov chain, P, with state space S=(1,2,...,N ). A stationary distribution o f a Markov chain satisfies m=mP- It is well known that an irreducible and aperiodic Markov chain has a unique stationary distribution. For large N the problem o f solving for m becomes intractable. However, there is a
useful way o f characterising the unique stationary distribution which is sufficient for our purposes.
A z-tree, h, defined on state space S, is a set o f ordered pairs, (/ j ) i j G S , such that each state i;^z is the initial point o f one arrow and from every state there is a path which leads to z. Denote the set o f all z-trees by H .
Then define the number u .
(/ •
h&H, ( i ^ j ) e h
N ow consider the directed graph, g, where each state i e S is the initial point o f one arrow and there is a unique loop which contains z. The set o f all possible graphs for state z is denoted .
Define the number,
r i4
g e G , 0 ^ j ) e g
The sets H, and G, are illustrated for the case S=( 1,2,3).
^ 2 ^ 3
t 2
^ 3
can be written in terms o f u^ as follows,
i * z i ^ z
That is we can either take each i-tree, i7^= z, and add the transition i ^ z or take each z-tree and add the transition z ^ i for all i z.
Hence
i * Z i * Z
u=Pu where u=( u,
1 “ /
j
So normalising the vector u gives us the unique stationary distribution.
2.1.2 Kandori Mailath and Rob.
This paper looks at the consequences o f introducing ongoing mutations into an evolutionary model. The main result is that the set o f equilibria is drastically reduced in the limit as the mutation rate goes to zero. Consider a finite population that is randomly matched each period to play the 2 x 2 symmetric game o f figure 2.1. Si S2 S] S2 A, A B ,C C ,B D .P , Figure 2.1
At the beginning o f every period each agent chooses a strategy that he uses for that period. The average payoff to a player using strategy Si, TCi, is equal to the expected payoff this strategy yields against a mixed strategy where Si is played with a probability equal to the proportion o f the remainder o f the population using s%. This average payoff is consistent with an infinite number o f random matches each period or with each player being matched exactly once with every other player in each period. Denote the state o f the system by the number of agents using Si, Zt. When agents adjust their strategy they adopt the strategy that yielded the highest expected payoff in the previous period. In a 2 x 2
Coordination Game this will result in convergence to either the state 0 or N depending on the initial point. Without mutations the system will then remain there forever. If we now allow agents to change their strategy with some positive probability 8 independently o f each other, then the system will no longer get stuck in one o f the equilibrium states. In fact, there will be a positive probability of going from any state to any other, as any number o f simultaneous mutations can occur. We therefore have an irreducible and aperiodic Markov process, P, on state space S = (0 ,1 ,....,N ). Each transition probability, Pÿ, is a polynomial in 8. We now make use o f the characterisation o f the unique stationary distribution given in section 2.1.1.
The value u^ is constructed by taking the product o f transition probabilities along each z-tree and summing this over all z-trees. Hence u^ is also a polynomial in 8. The stationary distribution is just a normalisation o f the vector u and is given
uAe)
by n (e) = ( H |(e ),... where H ,(£ ) =
We are interested in li m |i( 8 ) . Let the low est power o f 8 in u^ be and
define L = min L . z e S ^
If L > L then > 0 as 8 ^ 0
If L = L then > f as e - > 0 where 0 < / < I
Hence the limit distribution |X = lim)Li(8) will put a positive probability on E->0
o f the lowest power. Let be the lowest power o f 8 in Pj^. W e call this the cost of the transition i- ^ j , since it is the minimum number o f mutations required for the transition. The cost o f a z-tree, h, is the minimum number o f mutations required to move along it. This is given by c^ = ^ c.j . The low est power o f 8 in u^ will be
determined by the z-tree which has the low est cost. Thus L^= min c^.
heH^
Therefore L* will be determined by the state that has the lowest cost z-tree o f all states. So all we need to do to characterise the limit distribution is to find the state which has the lowest cost z-tree.
To illustrate the idea, consider the following example. A population o f 10 individuals are randomly matched to play the Coordination Game given in figure 1.2. The state o f the system at time t is given by the number o f agents playing s,, q^. Assume the dynamics are such that each period one agent revises his strategy: if seven or more agents play s,, the deterministic dynamics will move one place towards the state 10; if six or less play s,, the dynamics will move one place towards the state 0.
From any initial position the system will m ove to state 10 or 0 and then stay there. The introduction o f mutations allows the system to move between equilibria. Each individual changes his strategy independently and with probability 8. Hence we have an irreducible and aperiodic Markov chain, P. The transition probability P_ encompasses all the possible combinations o f mutations. For example, consider the transition 7 - > 6 . With no mutations the deterministic dynamics will take the system to state q,^j=8, where eight agents play s, and tw o play S;. If two o f the s, players mutate and none o f the s^ players, then the system
will move to =6. There are other ways o f moving to state 6. For example, we could have four o f the s, players and both o f the s^ players mutating. This requires six mutations. Note that in this case the smallest power o f 8 in is 2.
State 9 cannot have the minimum cost z-tree because it includes an arrow