Marco institucional del BANDES en Venezuela

In this section, we apply our ideas to a simple MJP, the M/M/c/c queue. In queuing theory (Kendall,1953), an M/M/c/k queue is a system consisting of c ‘servers’ and a queue of size k − c. A much studied instance of these systems is the M/M/c/∞ queue (abbreviated as M/M/c); here the queue is infinitely large. In this section, we focus on the M/M/c/c queue, which, despite its name, does not possess any queue. The ‘M’ terms indicate that the arrival process is Poisson and service times of each server is exponentially distributed (so that both processes are memoryless). For an M/M/c/c queue, individuals (customers, messages, packets, manufacturing jobs etc) arrive via a homogeneous Poisson process and are instantly handed to one of the servers; when no servers are free, they are discarded. The M/M/c/c queue is sometimes called the Erlang loss model (Medhi,2002) and has been used to model a variety of phenomena such as traffic in telephone networks, computer networks etc (Asmussen,2003).

Let α be the rate of the arrival process, and let the average service time of the servers be 1/β (remember that this quantity is exponentially distributed). Let S(t) represent the number of busy servers at time t. Then, under the M/M/c/c queue, the stochastic process S(t) evolves according to a simple Markov jump process on the space S = {1, · · · , c}. This MJP is a birth-death process whose state can change only by 1. When S(t) = s < c, a transition from s to s + 1 occurs with a rate α. On the other hand, a transition from s to s − 1 occurs with rate sβ. In our previous notation, the various transition rates Ass0 are:

Ass0 =      α s0 = s + 1, s < c sβ s0 = s − 1 0 otherwise (7.1)

A special instance of the M/M/c/c is the M/M/∞ queue, where the number of servers is infinite. This is sometimes called an immigration-death process (Asmussen, 2003). Individuals enter the population according to a homogeneous Poisson process whose rate is independent of the population size, while each individual has a fixed rate of dying (so that the rate at which the population decreases by 1 is proportional to the population size). This is often used as an approximation of the M/M/c/c queue with large numbers of servers, although it is an interesting model in its own right. Observe that since the number of active jobs in the M/M/∞ queue is unbounded, we cannot upper bound the event rates in the system (seeequation (7.1)). Thus, our uniformization-based MCMC sampler is not applicable to this system. Instead, one has to approximate the system

The M/M/c/c queue 119

with an M/M/c/c queue; this is how we proceeded when we faced a similar problem with the Lotka-Volterra model insubsection 4.3.1. By contrast, the leaving rate of any state s is sα + β. Since this is finite, we can apply our thinning based sampler.

In the following, we consider the evolution of an M/M/∞ queue over an interval [0, tend]. Our dependent thinning scheme ensures only a finite number of candidate state tran- sitions over this interval. Suppose that the state of the system was perfectly observed at time 0 to be s0. The birth-death nature of the process means that at the ith can- didate jump time wi, S(wi) can take a maximum value of (s0+ i). Thus in this case, the dimensionality of all messages is finite, allowing a straightforward application of the forward-backward algorithm. A complication arises when the initial state is noisily observed. If we allow the range of s0 to be infinite, then even if the number of steps in the forward-backward algorithm is finite, the dimensionality of each message is infinite. A simple way around this problem is to take a slice sampling approach (Neal, 2003a;

Walker,2007), instantiating only a finite number of states at any iteration.

Accordingly, associate a slice variable l with the initial distribution over states, and let it be uniformly distributed on the interval [0, 1]. Observe that

π0(s0) = P (l < π0(s0)) = Z 1

0

1(l < π0(s0))dl (7.2)

Thus, the joint probability of initial state s0 and l is given by

P (s0, l) = 1(l < π0(s0)) (7.3)

Given the state s0, we resample l uniformly on the interval [0, π0(s)]. Conversely, for a given value of the slice variable l, s0 is uniformly distribution over all states s such that π0(s) > l.

Let smax₀ be the largest state satisfying this condition:

smax₀ (l) = max s s.t. π0(s) > l (7.4)

Let l be the current value of the slice variable, so that smax₀ (l) is the maximum value of state at time 0. Then, the maximum value of the si, the state at step i of the forward- backward algorithm, is smax₀ (l) + i. We now can easily run the forward-backward algorithm to sample a new trajectory. At the end of this step, let ˜s0 be the new state at time 0; we then resample a new value of the slice variable ˜l as follows:

˜_{l ∼ U (0, π0(˜s)) (7.5)}

It it possible to introduce more slice variables for more control over the dimensionality of the MJP state space; however, we will not discuss such schemes here.

The M/M/c/c queue 120 1 2 5 10 20 0 10 20 30 40 50 60

Effective samples per second

Uniformization Dependent thinning Thinning (trunc) 1 2 5 10 20 10 20 30 40 50 60 70

Effective samples per second

per unit interval length

Uniformization Dependent thinning Thinning (trunc)

Figure 7.4: The M/M/∞ queue: (left) ESS per unit time, (right) ESS per unit time scaled by interval length.

7.3.1 Experiments

In the following, we considered an M/M/∞ queue, with parameters α and β set to 10 and 1 respectively. For some tend, the state of the system was observed perfectly at three times 0, tend/10 and tend, with values 10, 2 and 15 respectively. Conditioned on these, we sought the posterior distribution of the state trajectory on the interval [0, tend]. We compared our uniformization-based sampler from chapter 3 with the the generalized thinning-based sampler we outlined in this chapter. To run uniformization, we approximated the M/M/∞ system with an M/M/50/50 system. We also applied the thinning-based sampler to this truncated approximation, labelling it as ‘Thinning (trunc)’. All samplers were implemented in Matlab. For the uniformization-based sampler, we set Ω = 2, so that the subordinating Poisson process had a rate of 120. For the other two samplers, we set the thinning probability equal to a half, so that for any state s with leaving rate As, candidate leaving events were generated from a rate 2As process. The large state spaces involved makes particle MCMC very inefficient, and we did not include it in our results.

For all three samplers, we calculated effective sample sizes produced per unit time as we varied the interval length tend from 1 to 20. In all cases, we ran 10000 MCMC iterations with a burn-in period of 1000. At the end of any MCMC run, we calculated the number of state transitions, as well the amount of time spent in states 0 to 20. We estimated effective sample sizes for all these statistics, and summarized them with their median. The left plot in figure 7.4plots this for the three samplers as we varied tend. Sampling a trajectory on a long interval will take more time than on a short one, and to more clearly distinguish performance for large values of tend, the right plot in

figure 7.4scales the each result from the left plot with the length of the interval tend.

We see fromfigure 7.4that for short intervals, uniformization is significantly more in- efficient that our other two samplers. This is because the subordinating Poisson rate of 120 is much larger than the observed rates in typical trajectories sampled from the pos-

The effect of an unstable state 121

terior. Thus, a large number of the candidate transition times had to be thinned and the long Markov chains for the forward-backward algorithm resulted in long computation times. By contrast, the other two samplers produce much fewer candidate transition times, and therefore require much less computation time per iteration. Slower mixing notwithstanding, they perform much better. Interestingly, running the thinning-based sampler on the truncated M/M/50 queue offers no significant computational benefit over running it on the full model.

As the observation interval becomes longer and longer, the MJP can make larger and larger excursions (especially over the interval [tend/10, tend]). Thus as tend increases, the number of thinned events in all three samplers starts to become comparable. This, coupled with its faster mixing, causes the uniformization-based sampler to approach the performance of the other two samplers. At the same time, we see that the difference between the truncated and the untruncated samplers starts to widen. Of course, we should also remember that over long intervals, the effect of truncating the system size to 50 becomes more and more likely to introduce biases into our inferences.