9We make reference to Cressman [1996], Theorem 3.1, and the Lemma in Nachbar [1990].
r®(f)
^®It is important to notice that lim - p = L > 0 is only a necessary condition for the weakly dominated strategy to be in
(0
r^{t)
the support of the limiting distribution, whilst l i m - ^ = 0 is sufficient for extinction.
• Theorem 3.4.1. Any interior solution r(r(0),r) of a MS dynamic r = f{r { t)) converges to
Before proving the theorem, we need an additional lemma. We know from (3.4.10) that the relation is transitive. This result, as well as any other result proved so far, is not peculiar of the Centipede Game, as it holds for any finite normal form game. However, if we restrict our attention to the Centipede Game, we can can prove the following:
• Lemma3.4.1. For any interior solution, r(KO),r), of a MS dynamics r= f{r { t)) the relation
strictly orders the sets S, and S^.
Pro o f. The proof is by induction on 0 , as we will show that, for any i g 3 and 0 g { 1 ,...,//} strictly orders the sets ^ ( 0 ) = {-sf
Let J be the player who is required to move last. When G = N , ^ ( 0 ) = {jy and
c / ) ( 0 ) s and the lemma is obviously true, since is weakly dominated by j y. • S tep 1. 0= A - 1 . In this case, ^ ( 0 ) = {j'y ,jy^'} and = To prove the
lemma, it is sufficient to show that - \ ) can be ordered by <^, since ^ { N - \ ) = ^ { N ) . We evaluate the payoff difference [u_j(s‘^ j\r j( t) ) - u _ j(s ‘! j\r j(t) )) explicitly, factorising tj'it) out :
= « /(« .,(W +1) -
u_,(.N-1 ))-^ !^ -
-1) -
j
with, by (3.3.2.1-2), both { u _ j(N -l) -u _ j(N ) ) and {u_j(N+ l ) - u _ j ( N - I ) ) positive. Only the payoffs u _ j( N - l) , u_j{N) and M_y(A^ + l) enter in the evaluation of (3.4.16), since s’^j^ and yield the same payoff against any pure strategy in which J opts out before stage M- 1 . Moreover, the sign of (3.4.16) depends only on the sign of the term into round brackets of the right hand side of (3.4.16), since, by forward invariance, r f ( t ) > 0 , \ / t > 0 . Define x- >0 by K = that is, the threshold value of - that makes player -J
{u_j{N + \ ) - u _ j { N - \ ) y r^it)
indifferent between and ^îî'y '. Taking limits of (3.4.16) we obtain the following; limsign[w_y(/;', />(r)) - M _ y (//, />(0)]
r ,
\
10 4 ^ % )
= sign[((«_y(A^ + 1 ) - u_j(N - D)L^"' - {u_j(N - 1 ) - u_j(N))) Ihn (Oj
with l y = lim \ . There are only two possible alternatives: r^it)
• CASE A. > K. Then ' (with t = 0 and = jo-^ g | , since, by (3.4.7), ^ is continuously decreasing in t\.
0 vO
CASE B. Then (with t solving ^
• j
(0
= and
C, = i ( T, e A, S i r )
Since this exhausts all cases, the result follows.^i
• STEP 2. 1 < 0 < . Assume that the lemma is true for 1 < 0 < 7/. Let / be the player who is required to move at stage 0. When ! < 0 < N , ^ 0 ) = and (0) = r }- To prove the lemma, is sufficient to show that ^ , . ( 0 - 1 ) is an ordered set, since ^ ( 0 - 1 ) = ^ ( 0 ) . For a fixed 0, let û(Ô) index the pure strategy of player i which weakly r-dominates all other strategies in ^ ( 0 ) (i.e. s f s f for any s f g ^ ( 0 ) , with
e ' ^ û ) . W hen ! < 0 < N , for any g ^ ,.( 0 ) , the payoff d ifference
takes the following form:
- « - M t r
. -K O )=r f
X
- »). ‘ =
r
g I
(3.4.18) with all (u_i(k) - w_,(0 -1 )) strictly positive, except for (m_,(0) - u_.(0 -1 )) < 0. If ^ ( 0 ) is an ordered set, then (3.4.8) implies that ^ . (m_,(â: ) -m_,(0 - 1 ) ) must converge
r. (t) to some constant L®. There are two possible cases:
• L® > (<)0. Then jf:' (jfj / : ' ) , by anakogy with CASE B.
• L® = 0. Then jf:' j!'. . To show this, notice that the result is obviously true if è = 6 , since this would make (/) strictly decreasing in t.
Assume instead # # 0 . Then, it must be:
lim rfitX
lim
e"€^(.e)\e
(«.,(0
(0))
>0
. (3.4.19)might be argued that we are not allowed to determine the limiting sign of (3.4.17) looking only at the term into round brackets of the right hand side, since (t) might go to zero fester than
out by the feet that, by (3.4.7), L y ’*’' is finite.
r^(t)
Given that converges to a positive constant, (3.4.10) implies that:
C7_,.) = for any C7_,. e co_,(r(G)). (3.4.20)
In other words, the relative performance of sf must eventually improve, compared with any other strategy in ^ ( 9 ) . Thus, for t sufficiently large, also jf:' must improve compared to since sf is the only strategy in u>f{9) against which af:' does better than j!'.. From the above consideration we have that ^ f(t) must converge to 0 from above, which in turn implies, by analogy with CASE A, / j .
Since this exhausts all cases, the result follows. # We are now in the position to prove Theorem 3.4.1.
• Proof OF Theorem 3.4.1. Since S, and S„ are strictly ordered by <^, (3.4.8) implies that lim exists for any i,9 and 9'. If all the ratios converge, then also the mixed strategy profile must converge. We can therefore apply the standard result “convergence implies Nash” in the case of MS dynamics (see, e. g. Weibull [1995], Theorem 5.2 (c)) to complete the proof.#
Another way to rephrase the content of Theorem 3.4.1 and its corollary could be the following. The theorem shows that the Nash equilibrium component denoted globally interior attracting that is, it attracts every interior path under any MS dynamics. Note that the above result is not directly linked with any local stability property of the s e t v ^ it may well happen that trajectories starting close t o . / ^ move away and then, eventually, come back. This is exactly what happens in the Centipede Game. In the next section, we shall explore this phenomenon through simulations.
3 .5 . «Cycles OF LEARNING»: somesimulationresults
This section is devoted to simulations of the dynamics studied hitherto. To perform this task, we shall begin by specifying the payoff structure, as well as the dynamics:
• Assumption 3.5.1. The payoff function is as follows:
Mf(9) = ^ > 1; (3.5.1)
u„{9) = Uj{9 + \) (3.5.2)
In words: the payoff of player i is multiplied by some positive constant a after every other round. We can therefore interpret a as a measure of the increasing returns to cooperation in the Centipede Game, since it reflects how much the payoffs increase as the game ends further
away from the beginning of the tree. The assumption for the dynamics (which replaces Assumption 3.3.1) is the following:
• Assumption 3.5.2. The evolution of r{t) is given by the following system of continuous-time differential equations:
rf{t)= rf{t)(ui(j*, r_,.(0) - M,(r(f))) + A,.(/If - rf{t)) (3.5.3)
with A > 0 and p f €(0,1). These dynamics are a linear combination between the standard Replicator Dynamics and a perturbation term which ensures that, at each point in time, every pure strategy is played with positive probability, no matter how it performs against the opponent’s mixed strategy. Following Binmore and Samuelson [1995] we call the latter drift. Such a deterministic perturbation term can serve as a high probability approximation to a stochastic noise term in a model in which time is discrete and the population size is finite^^, as we approach the limiting case of continuous-time and infinite population suitably. The relative importance of the drift is measured by A^, which we refer to as the drift level. We assume A, to be “small”, since the major forces which govern the adjustment process should be captured by the unperturbed dynamics.
We shall analyze the 2-legged Centipede Game of Figure 3.5.1 first.
I I
4
U ’J
a
y 1 ( i - y ) 3 ( l - x ) X 2 3 a , l a , \ W Figure 3.5.1A 2-legged Centipede Game.
strategies are played and Æ ’ = ] (x,y) e A y = 1,XG 0,-
In Figure 3 5.1 x and y denote the probabilities with which the corresponding pure
J ■
- I - a -1-1.
In the phase diagrams of Figure 3.5.2 we trace some interior solutions of the unperturbed Replicator Dynamics (i.e. when X, = Ay/ = 0) under two different realizations of the payoff parameter a .
^^Model which fail in this category are, for example, those of Kandori e t a l . [1993] and Young [1993]- In a biological context, this noise may be interpreted as a m u ta tio n , i. e. a random alteration of the agents’ genetic code. In a learning context, it can be interpreted as a mistake, i. e. a random alteration of the agents’ behavior, or as an effect of the players’ experimentation. We prefer the terminology of drift (as opposed to noise) because the latter is usually modeled as a genuine random variable, whereas the former takes the form of a purely deterministic dynamics. For a general discussion on motivations and general properties of evolutionary dynamics with drift, see Samuelson [1997], Chapter 6.
Figure 3.5.2 gives a good description of the content of Propositions 3.4.1-2. First note that (t')
weak domination of s], implies that and therefore x , is strictly decreasing in t , for any ^//(O
f>0. Whenever x falls below ■ , the threshold level of x which makes player /
+ a + \
indifferent between her two pure strategies, rf(t) starts to fall, until it vanishes in the limit. For any interior solution, is in fact a strictly r-dominated under the Replicator Dynamics (even if it is not strictly dominated in the conventional sense) and this implies, as we know from Proposition 3.4.2, lim/f(r) = 0. 0 0 0 0 0 . 2 0 . 4 0 . 6 0 . 8 1 0 . 2 0 . 4 0 . 6 0 . 8 1 a) a = 2 b) a = 4 Figure 3.5.2
The 2-legged Centipede Game and the Replicator Dynamics.
As the (expected) payoff difference between s i and s], tends to zero, as both strategies yield the same payoff against s].In consequence, the evolutionary pressure against the weakly dominated strategy s],vanishes, and this is why s], remains in the support of the limiting play.
Consistently with Theorem 3.4.1 and its corollary, every interior trajectory converges to the corresponding Nash-equilibrium component Æ ’, highlighted by a bold segment in the upper- left corner of the two diagrams. Increasing the payoff parameter a has the following effects:
• ./^ sh r in k s, i.e. the measure of states compatible with the Nash prediction is reduced; • the dynamics speed up (this is because in the Replicator Dynamics, as well as in any MS,
growth rates are increasing functions of payoff differences).
It is interesting to note that both effects are qualitatively consistent with McKelvey and Palfrey [1992] ’s experimental results on the Centipede Game, for which our model may provide a theoretical account.
compare the two sessions of comparable length and number of observations characterized by a “LOW” payoff treatment (Sessions 1 and 3), with the unique session characterized by a “HIGH” payoff treatm ent (Session 4) of McKelvey and Palfrey’s
We move on to the 3-legged Centipede Game of Figure 3 5 3.
/
I I
■ > • •-\ o|
r « i r n u i F ig u re 3.5 . 3A 3-legged Centipede Game