Banca para el desarrollo de Venezuela

Consider a Markov jump process with state-space S. We allow the cardinality of S to be countably infinite. We will follow the notation of the previous chapter (rather than

chapter 3), so that Ass0 gives the rate of transitioning from state s to s0 for all s and s0. In particular, since we are dealing with MJPs, Ass = 0 ∀s. For any state s ∈ S, the leaving rate is a constant As =Ps0_∈SAss0. We require A_s to be finite for each s, however unlikechapter 3, we will not require a finite constant Ω > As, ∀s. Upon leaving state s, the probability of transitioning to state s0is ps(s0) ∝ Ass0, with p_s(s) = 0. Given an initial distribution over states π0, Gillespie’s algorithm (algorithm 3.1) provides a simple and direct way to sample a trajectory of this system over an interval [tstart, tend].

Dependent thinning for MJPs 115

Figure 7.1: Gillespie’s algorithm for MJPs (with auxiliary Poisson events)

In chapter 6, we described a thinning-based alternative to Gillespie’s algorithm. For each As, define a constant Us> As. Us gives the rate at which candidate leaving times are generated in state s, so that the time until the next candidate time is sampled from an exponential with rate Us. We reject each such event with probability

 1 −As

Us 

, otherwise we transition to state s0 with probability ps(s0). Note that since we are dealing with a Markov system, we do not need to represent the duration for which the system has been in its current state. This makes the entire procedure considerably simpler that that for general semi-Markov processes (chapter 6).

The Markov structure of the problem allows us to define an equivalent construction in terms of a family of Poisson processes, one for each state. This is demonstrated in Figure 7.1. For each state s, sample a realization of a rate As Poisson process on [tstart, tend]. Assign all events of this process the label s. Now, to sample a trajectory, assign the MJP an initial state drawn from the prior π0. Suppose we pick state s0, then the MJP remains in this state until the first event labelled s0. By the memoryless property of the Poisson process, this waiting time is exponentially distributed with rate As0, as required by the definition of the MJP. At this time, the MJP moves to a random new state s1, with ps0(s1) ∝ As0s1. Repeat the procedure until the end of the interval. Clearly, this procedure is equivalent to to Gillespie’s algorithm. The cyan-shaded region of figure 7.1 then defines the MJP trajectory (S, T ). The times of all events in this regions define T , while their corresponding labels define S. By the independence property of the Poisson process, everything outside this region is

Dependent thinning for MJPs 116

Figure 7.2: Thinning based construction for MJPs (with auxiliary Poisson events)

irrelevant.

We can now introduce auxiliary thinned variables by an obvious application of the thinning theorem. For each state s, sample events from a Poisson process, now with the rate Us > As on [tstart, tend]. To sample a trajectory, once again assign the MJP an initial state s0 drawn from π0. Once again, the MJP remains in this state until the first event labelled s0, however now it changes state only with probability As0/Us0. If it does decide to change state, it picks a new state s1 with probability proportional to As0s1. Again, repeat the procedure until the end of the interval. It is clear that this procedure is equivalent to the dependent thinning scheme outlined earlier (and in

chapter 6). Figure 7.2demonstrates this graphically; once again, the events inside the cyan region define (V, W ) the thinning representation of the MJP. Like chapter 3, we no longer need the set of waiting times L, since the original system is now Markov without self-transitions.

It is now easy to understand the MCMC sampler of the previous chapter. Recall that this proceeded by alternately resampling the thinned events given the MJP trajectory, and then the trajectory given the set of candidate transition times. Knowing the MJP trajectory amounts to knowing the the cyan region infigure 7.2(as well as the events at the right edges corresponding to actual transitions). Resampling the thinned events

Dependent thinning for MJPs 117

Figure 7.3: Resampling the MJP trajectory

in the interior of the cyan region is now a simple application of the corollary to the thinning theorem (corollary 2.1): when in a region corresponding to state s, sample from a Poisson process with constant intensity (Us− As). Proposition 6.2 shows that this is correct.

Resampling a new trajectory given the set of candidate times involves discarding the old labels of the Poisson events in the cyan region, and relabelling the events using the forward-backward algorithm. This is shown in figure 7.3. Note that we need to account for the probability of the new labels assigned to the Poisson events; we saw in the previous chapter how we to adapt the forward-backward algorithm to do so.

By assigning labels to candidate jump times, we associate each segment of the MJP trajectory with a window of events from the Poisson process labelled with the corre- sponding state. Since each of these Poisson processes has a finite rate, we will have only a finite number of candidate state transitions over any finite interval. Even if the maximum event rate in the system is unbounded, any realization of the system trajectory will have a finite maximum rate.

By contrast, uniformization involves constructing a MJP from a single subordinating Poisson process. In order to avoid assigning labels to these Poisson events, we need its rate to dominate all event rates in the system, something which is not always possible. However, since the Poisson process is independent of the system trajectory, there is a smaller dependence across samples. Note though, that our new algorithm requires only a slight modification of the uniformization-based sampler. It samples the thinned events

The M/M/c/c queue 118

from a slightly different Poisson process, and has a single additional term P (∆wi|vi) in the forward-backward algorithm.