To better understand what might happen at steady- state, let us assume that the master equation (5) has a unique stationary solution pX(xxx) := limt→∞pXXX(xxx;t) that is independent of the initial state. The probabil- ity distributionpXfXX(exxx;t) of the population density process
e X
XX(t; Ω) =XXX(t)/Ω will be given bypXXXf(xexx;t) = ΩpXXX(Ωxxex;t)
and will depend on the size parameter Ω in general. Let us define the function
V(xxex; Ω) :=−1
Ωln
pXXfX(xexx) pfXXX(xxex∗)
, (55)
wherepXfXX(exxx) := limt→∞pXXfX(xexx;t) is the steady-state distri-
bution of the population density process andxxex∗is a state
at which the stationary probability distributionpXXXf(xxex) at-
tains its maximum value. Note thatV(xexx; Ω)≥0. More- over, and as a consequence of Eq. (55), we have that
pXXXf(xxex) = 1 ζ(Ω)exp n −ΩV(xexx; Ω)o, (56) where ζ(Ω) :=X uuu expn−ΩV(uuu; Ω)o. (57)
In this case,pXXXf(xexx) is a Gibbs distribution with “potential
energy” function V(xxex; Ω), “temperature” 1/Ω, and par- tition functionζ(Ω). Clearly, V(xexx; Ω) assigns minimum (zero) potential to the states of maximum probability at steady-state and infinite potential to the states of zero probability.
We will now assume that, around the thermodynamic limit, the potential functionV(xxex; Ω) is an analytic func- tion of Ω−1. Then, a Taylor series expansion with re- spect to Ω−1 around zero (i.e., around the thermody- namic limit) results in
V(exxx; Ω) =V(xxxe;∞) + 1 Ω ∂V(xxxe;∞) ∂Ω−1 +· · · =V0(xxex) + 1 ΩV1(xexx) +· · ·, (58) where V0(xexx) :=V(exxx;∞) =− lim Ω→∞ 1 Ωln pXXfX(xexx) pfXXX(xxex0) ≥0, and V1(xxex) := ∂V(xxxe;∞) ∂Ω−1 .
As a consequence of Eqs. (56)–(58), and for sufficiently large Ω, we have that
pXXXf(xexx)' 1 ζ(Ω)exp n −ΩV0(xxxe)−V1(exxx) o ,
where the partition function is now given by ζ(Ω) =X u u u expn−ΩV0(uuu)−V1(uuu) o .
Therefore, the stationary probability distributionpXfXX(exxx)
satisfies a large deviation principle with rate func- tion V0(xexx) (Touchette, 2009). It turns out that, ifχχχ(t)
satisfies the macroscopic equations (44), then
dV0(χχχ(t)) dt = X n∈N ∂V0(χχχ(t)) ∂χn(t) dχn(t) dt ≤0,
with equality if and only if
dχn(t)
dt = 0, for every n∈ N.
Therefore, the solution χχχ(t) of the macroscopic equa- tion (44) produces a downhill motion in the value of the potential energy functionV0 (which is a Lyapunov func-
tion for the macroscopic system) until it asymptotically reaches a stable stationary state.
For a given value of Ω, we can view the multidi- mensional surface V0(xxex; Ω) as a potential energy land-
scape(Aoet al., 2007; Wanget al., 2008, 2011; Zhou and Qian, 2011). The stable stationary states of Eq. (44) correspond to potential wells (basins of attraction) as- sociated with the (local or global) minima of V0, which
are separated by barriers corresponding to hills (unsta- ble states) and saddles (transitional states – states on the potential energy surface from which stable states are equally accessible). Which minimum will be reached de- pends on the initial condition that must be a point in the potential well of that minimum. It turns out that, once the macroscopic system, given by Eq. (44), reaches a stable stationary state, it will stays there forever. As a consequence, the macroscopic system will only move downhill on the potential energy landscape.
It is now not difficult to see that Eqs. (56)–(58) imply that lim Ω→∞pXfXX(exxx) = lim Ω→∞ expn−ΩV0(xexx) o expn−V1(xexx) o P u uuexp n −ΩV0(uuu) o expn−V1(uuu) o = exp{−V1(xxex)} P u uu∈G∗exp{−V1(uuue)} , if xexx∈ G∗ 0, otherwise,
where G∗ is the set of allground states (global minima)
of the potential energy landscape V0 (i.e., the set of all
states at which V0 = 0). As a consequence, only the
ground states have a non-negligible probability to be ob- served as Ω becomes large because pXfXX(xxxe) decays expo-
nentially with Ω, for every xxex /∈ G∗. This implies that,
in the thermodynamic limit, the master equation (5) will asymptotically converge almost surely to a ground state of the potential energy functionV0, independently of the
initial state. The particular ground state is chosen with probability determined by the values of the potential en- ergy function V1 over the ground states of V0. On the
other hand, the macroscopic equation (44) will reach a minimum ofV0, which may or may not be a ground state
in G∗ depending on the initial condition. If the macro-
scopic equation has a unique asymptotically stable sta- tionary solution that is independent of the initial condi- tion, then V0 will have only one (global) minimum. In
this case, and as we mentioned before, the master equa- tion (5) will converge almost surely to the same state in the thermodynamic limit. However, ifV0 contains more
than one minimum, then the stationary solution of the master equation (5) may be different from the station- ary solution predicted by the corresponding macroscopic equation (44). As a consequence,
lim
Ω→∞t→∞lim pXXfX(xexx;t)6= lim
t→∞Ωlim→∞pXXXf(xexx;t)
in general. This distinct difference between the station- ary behavior of the master equation (left-hand side of inequality) and macroscopic equation (right-hand side of inequality) is known asKeizer’s paradox (Keizer, 1987; Qian, 2010; Vellela and Qian, 2007).
At finite sizes Ω, the modes of the stationary proba- bility distributionpXXfX(xexx) (i.e., the most probable states)
correspond to the minima of the potential energy land- scapeV(xexx; Ω). Ifxxxe0 is a minimum ofV0(xxex) + Ω −1V 1(xxex), then V0(xexx) ≥ V0(exxx 0 )−Ω−1 V1(xexx)−V1(xxex 0 ) , for every e x
xx ∈ W(xexx0), where W(xxex0) is a local neighborhood of xxxe0
for which the inequality is satisfied. For large enough Ω, such that max e x x x0 max e xxx∈W(exxx 0) V 1(xxxe)−V1(exxx 0 ) V0(xxxe 0 ) Ω<∞,
where the first maximum is taken over all minima of
V0(xxex) + Ω
−1V
1(xxex) which are not ground states ofV0(xxex),
we haveV0(exxx
0
).V0(exxx), for everyxexx∈ W(xexx
0
), and there- forexxex0 will approximately be a local minimum of the po- tential energy landscapeV0. In this case, the peaks of the
stationary probability distributionpfXXX(xxex) will correspond
to stationary states of the macroscopic equation (44).12 For this reason, we can refer to the peaks in p
e
X(xxex) as
macroscopicmodes.
At smaller values of Ω, the stationary probability dis- tributionpXfXX(exxx) will be given by Eqs. (56) and (57). The
modes will now depend on the fluctuation size parame- ter Ω and will be determined by the minima of the po- tential energy landscapeV(xxex; Ω). However, a state that
12Note that the converse of this result is not necessarily true. There
might be stationary states of the macroscopic equation that, for a given value of Ω, do not introduce peaks in the stationary probability distribution. To see this, recall that, in the limit as Ω→ ∞, the only peaks present in the stationary probability distribution are the ones associated with the global minima ofV0.
minimizes the potential energy functionV0may not nec-
essarily minimizeV, in which case at least some modes of the probability distribution pXXfX(xexx) will not be predicted
by the corresponding macroscopic equation. Since these modes show up at small system sizes, in which apprecia- ble stochastic fluctuations may be present in the system due to “intrinsic noise,” we refer to them asnoise-induced
modes. Recent literature has documented the presence of noise-induced modes in biochemical reaction networks and their importance in modeling system behavior not accounted for by their macroscopic counterparts (Arty- omov et al., 2007, 2009; Bishop and Qian, 2010; Qian, 2010, 2011; Qianet al., 2009; Zhanget al., 2010c).
We finally note that, if a Markovian reaction network is at a stable statexxexs1 at timet0, then it may switch to a
another stable statexexxs2at timet0< t <∞with probabil-
ity peΩ(xxxe s 2;t). However, limΩ→∞peΩ(exxx s 2;t) =δ(xxex s 2−χχχ(t)),
where χχχ(t) is the solution of the macroscopic equa- tions (42), initialized with xexxs1. Since exxx
s
1 is a minimum
of the potential energy function V0, the macroscopic
system will be in state χχχ(t) = xexxs1 at time t. Hence, limΩ→∞peΩ(xxex
s
2;t) = 0. As a consequence, the proba-
bility of switching from a stable state to another stable state tends (in general exponentially) to zero as the sys- tem size increases to infinity. At finite system sizes Ω, switching among stable stationary states becomes pos- sible, but the probability of switching is very small for large Ω; i.e., switching among stable stationary states arerareevents (Qian, 2011; Zhou and Qian, 2011). As a matter of fact, we can approximate the waiting time for switching by an exponential distribution (Aldous, 1982) with rate parameter that tends to zero as Ω→ ∞.