Require: Transition kernel T, and the MGFs Eτ
exp s maxn∈[N]Sn,i
, Eτ exp −sA1 1: ifA1andA2andA3andA4then
2: Transform T to get ˜L(c, dτ; s)(see (4.1.5))
3: exp ζ(s)←maximal eigenvalue of ˜L(c, dτ; s) 4: θ←sup{s>0|ζ(s)≤0}
5: Normalise r(., θ)so that E[r(C0, θ)] =1 6: ϕ(θ)←sup
c∈E{1/r(c, θ)} 7: end if
8: returnThe decay rate θ and the prefactor ϕ
In the following sections, we apply our results to several application scenarios. They are intended to serve as illustrative examples. For the sake of simplicity, assume that the state spaceE of the chain{Ck}k∈N0 is finite. Then, the transition kernel T of{Ck}k∈N0
is just a transition matrix. Let us write T= ((ti,j)). We do allow the servers to follow dif- ferent probability distributions satisfying stability conditionsA3andA4. For purposes of illustration, we consider exponentially distributed service and inter-arrival times in the following examples.
4.3 f o r k-join system with non-renewal input
In this section, we describe an FJ system with Markov-modulated inputs. This is prin- cipally motivated by recent empirical evidences that reveal burstiness in Internet traffic and also in inputs to MapReduce clusters (Y. Chen, Alspaugh, and Katz 2012; Heffes
and Lucantoni1986; Kandula et al.2009; Yoshihara, Kasahara, and Takahashi2001). In
general, in order to model this burstiness, we can assume the inter-arrival times to be modulated by some Markov chain{Ck}k∈N0.
Example 4.3.1(Markov-modulated inter-arrival times). Suppose the modulating Markov chain takes four distinct values (corresponding to different phases of arrival traffic). In state j of the chain, suppose the inter-arrival times are exponentially distributed with parameter λj. Also assume, the service times at the n-th server are exponentially dis- tributed with parameter µn. Then, the transformation in (4.1.15) is given by
t1,1 t1,2 t1,3 t1,4 t2,1 t2,2 t2,3 t2,4 t3,1 t3,2 t3,3 t3,4 t4,1 t4,2 t4,3 t4,4 7→ t1,1λλ1+s1 t1,2λλ2+s2 t1,3λλ3+s3 t14λλ4+s4 t2,1λλ1+s1 t2,2λλ2+s2 t2,3λλ3+s3 t2,4λλ4+s4 t3,1λλ1+s1 t3,2λλ2+s2 t3,3λλ3+s3 t34λλ4+s4 t4,1λλ1+s1 t4,2λλ2+s2 t4,3λλ3+s3 t44λλ4+s4 .
Having done the above transformation, the decay rates are found as
θn =sup{s>0|
µn
µn−sχA(s)≤1},
θ=sup{s>0|β(µ; s)χA(s)≤1},
Figure 4.3: Numerical verification of the bounds (shown in darker shade) for blocking systems. (Top) An FJ system with Markov- modulated arrivals. The modulating Markov chain takes values in the setE= {1, 2, 3}. The exponential inter-arrival times have parameters 0.25, 0.4, and 0.50. (Middle) An FJ system with Markov-modulated service times. The mod- ulating Markov chain takes values in the set
E = {1, 2, 3, . . . , 32}. The exponential inter- arrival times have parameter 0.35. (Bottom) An FJ system with Markov-modulated arrival and service times. The modulating Markov chain takes values in the setE = {1, 2, 3, . . . , 64}. In all the cases, there are five heterogeneous, block- ing servers whose service rates are drawn ran- domly, satisfying the stability conditions inA3 and A4. The transition probabilities and the initial distribution of Ck are chosen randomly. Observe that the analytic bounds obtained in Corollary4.2.1 are in close agreement with the sample estimates of the tail probabilities of the waiting times. Waiting time Probability 0 20 40 60 80 100 10 −6 10 −4 10 −2 10 0 Waiting time Probability 0 20 40 60 80 100 10 −6 10 −4 10 −2 10 0 Waiting time Probability 0 20 40 60 80 100 10 −6 10 −4 10 −2 10 0
where χAis the largest eigenvalue of the transformed matrix and
β(µ; s):=
∑
S∈{A⊂[N]|A̸=∅}
(−1)|S|+1 (∑i∈Sµi) (∑i∈Sµi)−s
, (4.3.2)
is the MGF of the maximum of N independent exponential random variables (see Ap- pendix B.2). After normalisation of the right eigenvector, one obtains the bounds us- ing (4.1.13) and (4.2.2) for the work-conserving and the blocking system respectively. Please see Figure4.2 (for work-conserving systems) and Figure 4.3 (for blocking sys- tems) to compare our bounds against CCDFs obtained from Monte Carlo simulations.
4.4 pa r a l l e l s y s t e m s w i t h d e p e n d e n t s e r v e r s
In this section, we consider an FJ system as described in Section4.1 with correlated servers. To be precise, we assume that the service times are modulated by a Markov
Figure 4.4: Single-node FJ system with a provisioning where the heavier part of each incoming job is apportioned in a round-robin fashion. Let Jobi = (Jobi,1, Jobi,2), where Jobi,1denotes the heavier sub- job. For instance, for the first job Job1, the sub-job Job1,1 is allotted to servers 1, 2, . . . , d and Job1,2, to the rest. Then, Job2,1is allotted to servers d+ 1, d+2, . . . , 2d and Job2,2 to the rest, and so on.
d+2 d+1 . . . . . . 1 2 d N Job arrivals
Service time distributions appropriate of the heavier sub- job assigned here.
Service time distributions appropriate of the lighter sub-job assigned here. Ai Jobi,1
Jobi,2
chain. The motivation behind this is the phase-type behaviour that service times show due to various exogenous effects. Before furnishing numerical examples, we mention some factors that might engender such a phase-type behaviour.
u n e q ua l j o b s i z e s Phase-type behaviour may arise when the sizes of the incoming jobs are unequal enforcing a change of service time distribution across the servers. In- tuitively, heavier jobs demand greater service times in total. This can be modelled by scaling up the service times or the parameters of their distributions whenever a heavier job arrives. For instance, in the context of MapReduce, the job sizes can be time vary- ing. In the context of Multi-path TCP, the packet sizes are usually of different sizes. The modulating chain captures the different job sizes enforcing different service time distributions. The state space of the chainE can be chosen depending on the particular application under consideration.
p r ov i s i o n i n g i n m a p r e d u c e The “irregular” service times may also arise due to provisioning, even when the job sizes are constant. Suppose that the incoming jobs are split unequally among the available servers. The rule that decides job division into tasks is termed provisioning. Such provisioning can be employed in MapReduce systems to influence waiting times. Consider the following example: Each job consists of two sub-jobs one of which is more demanding than the other. That is, Jobi = (Jobi,1, Jobi,2), where Jobi,1 can be assumed to be heavier (more time-consuming) without loss of gen- erality. Now, in order to apportion the burden of the heavier job, devise a variant of round-robin mechanism such that for the first job Job1, the sub-job Job1,1 is allotted to servers 1, 2, . . . , d and Job1,2, to the rest N−d servers. Then, Job2,1 is allotted to servers d+1, d+2, . . . , 2d and Job2,2to the rest, and so on. Mathematically this is equiv- alent to having a modulating Markov chain that starts at state 1 where it assigns service time distributions appropriate of the heavier job (e.g., scaled service times as explained before) to servers 1, 2, . . . , d and the usual unscaled service time distribution, to the rest, and then jumps with probability one to state 2 where it assigns service time distribu-
tions appropriate of the heavier job to servers d+1, d+2, . . . , 2d and the usual, to the rest. See Figure4.4for a pictorial description of this provisioning.
m o d u l at i o n i n m u lt i-path tcp Packet scheduling or load balancing mechanisms (Frömmgen et al.2017) could also give rise to correlated service times. The load balanc-
ing algorithm typically decides on the amount of packets to send over each path with the objective of keeping congestion under control. Taking the liberty of mathematical ab- straction, we can model such a scenario with a Markov chain (representing the decisions of the load-balancer) that modulates only the service times of the system.
e f f i c i e n c y d i f f e r e n t i at i o n Servers may themselves have their own high and low efficiency periods that may or may not depend on the state of the other servers, e.g., enforced by energy-saving routines (Sharma et al.2013). The service rates may also
be modulated by the user. For instance, given a fixed monetary budget, the user of a cloud computing service such as the Amazon AWS, may be forced to switch to a less expensive machine (with inferior service rates) when the price of the current machines increase, to meet the budget constraint (e.g., see Shastri, Rizk, and Irwin (2016)).
Example 4.4.1 (Markov-modulated service times). Motivated by the above scenarios, we now demonstrate the bound computation in (4.1.13) and (4.2.2). In the following example, assume the arrival process is renewal and inter-arrival times are exponentially distributed with parameter λ.
Suppose there are two servers each of which has two efficiency phases, high and low. We model this by two Markov chains modulating the servers, each on state space
{0, 1}. For the sake of simplicity, assume that server i is exponentially distributed with parameter µi or κi according as its modulating Markov chain is state 0 or 1. The two Markov chains may not be independent. Mathematically this is equivalent to having one single modulating Markov chain on state space{0, 1} × {0, 1}. We rename the states as
(0, 0)7→1,(0, 1)7→2, (1, 0)7→3, (1, 1)7→4.
Let us first look at the work-conserving system. For the 1st server, following (4.1.15), we transform t1,1 t1,2 t1,3 t1,4 t2,1 t2,2 t2,3 t2,4 t3,1 t3,2 t3,3 t3,4 t4,1 t4,2 t4,3 t4,4 7→ t1,1µµ1−s1 t1,2µµ1−s1 t1,3κ1κ−s1 t14κ1κ−s1 t2,1µµ1−s1 t2,2µµ1−s1 t2,3κ1κ−s1 t24κ1κ−s1 t3,1µµ1−s1 t3,2µµ1−s1 t3,3κ1κ−s1 t34κ1κ−s1 t4,1µµ1−s1 t4,2µµ1−s1 t4,3κ1κ−s1 t44κ1κ−s1 .
Transformation for the 2nd server is analogous. Denote the largest eigenvalues of these two transformed matrices by χ(1)S and χ(2)S respectively. The transformation for the blocking system is as follows
t1,1 t1,2 t1,3 t1,4 t2,1 t2,2 t2,3 t2,4 t3,1 t3,2 t3,3 t3,4 t4,1 t4,2 t4,3 t4,4 7→ t1,1β(µ1, κ1; s) t1,2β(µ1, κ2; s) t1,3β(µ2, κ1; s) t1,4β(µ2, κ2; s) t2,1β(µ1, κ1; s) t2,2β(µ1, κ2; s) t2,3β(µ2, κ1; s) t2,4β(µ2, κ2; s) t3,1β(µ1, κ1; s) t3,2β(µ1, κ2; s) t3,3β(µ2, κ1; s) t3,4β(µ2, κ2; s) t4,1β(µ1, κ1; s) t4,2β(µ1, κ2; s) t4,3β(µ2, κ1; s) t4,4β(µ2, κ2; s) .
Call its largest eigenvalue χS. The function β is as defined in (4.3.2). Having done the above transformation, the decay rates are found as
θn=sup{s>0| λ λ+sχ (n) S (s)≤1}, θ=sup{s>0| λ λ+sχS(s)≤1}. (4.4.1)
After normalisation of the right eigenvector, one finds the bounds on the tail probabilities of the steady-state waiting times using formulas in (4.1.13) and (4.2.2) for the work- conserving and the blocking system respectively. To see the quality of our bounds on a bigger state space, we simulated an FJ system with five heterogeneous servers being modulated by a chain having 32 states. See Figure 4.2 (for work-conserving systems) and Figure4.3(for blocking systems) to compare our bounds against empirical CCDFs.
4.5 m a r k ov m o d u l at e d a r r i va l s a n d s e r v i c e
In this section, we describe a system where service and inter-arrival times are dependent. This is essentially a generalization of Section4.3 and Section 4.4. All the motivating examples listed in Section4.3 and Section4.4 can be extended to this case to account for generalised application scenarios. While this allows us to endow service times of each server, and the arrival process, separate modulating Markov chains (which can be modelled by one single chain on the Cartesian product space as shown before), we can use this formalism to devise more advanced provisioning by taking into account the current job arrival rate (i.e., set efficiency of servers to “high” during busy period and to “low” otherwise etc.). This paves way for what we call “reactive provisioning.”
4.5.1 Reactive provisioning
We propose to take into account information on the current FJ system environment, e.g., estimates of the arrival intensities, and then modulate, i.e., set service rates accordingly. Such a provisioning is reactive in nature and hence the nomenclature. The changing en- vironment is essentially captured through the modulating Markov chain for the arrivals in this case.
Example 4.5.1(Markov-modulated inter-arrival and service times). Consider a Markov chain {Ck}k∈N0 capturing the changing environment in the sense that at state j of the
Markov chain, the inter-arrival times are exponentially distributed with parameter λj and accordingly, the service times at the n-th server are distributed exponentially with parameter µn,j. Define µ(j) := (µ1,j, µ2,j, . . . , µn,j). Then, the required transformation for work-conserving systems is tij → tij µ n,j µn,j−s λ j λj+s
, for the n-th server, and likewise, the transformation for the blocking system is given by tij →tijβ(µ(j); s)
λj
λj+s
. Let us denote the largest eigenvalue of the transformed matrix for the n-th server by χ(n)AS, and
that of the transformed matrix for the blocking system by χAS. Therefore, the decay rates are found as
θn =sup{s>0|χ(n)AS(s)≤1}, θ=sup{s>0|χAS(s)≤1}.
(4.5.1)
After normalisation of the right eigenvectors, we compute the bounds on the tail proba- bilities of the steady-state waiting times using formulas in (4.1.13) and (4.2.2). To see the quality of our bounds, we simulated the system with the modulating chain having 64 states. See Figure4.2for work-conserving systems and Figure4.3for blocking systems to compare our bounds against empirical CCDFs.
4.6 f u r t h e r e x t e n s i o n s
In the following, we discuss how the results obtained in this chapter can be further extended to cover wider application scenarios.
4.6.1 Design of Proactive Mechanisms
Markov-additive processes are capable of modelling not only reactive but also proac- tive systems. In Section 4.5, we modelled the changing environment with a Markov chain {Ck}k∈N0 and devised a reactive mechanism. For many applications, reactive
mechanisms may be expensive, and it is profitable to be able to anticipate the changes in the environment and act accordingly (e.g., set the service rates). Our Markov-additive process framework allows for such a proactive provisioning (see Figure4.1). In this cou- pled model, the distribution of the increments Xn,k+1A , for each n∈ [N], will also depend on Ck. Such a provisioning is promising as it allows for a notion of agility and adap- tation in parallel server systems. The preparedness aimed for in proactive provisions could potentially reduce cost and yield a smoother transition.
4.6.2 Replication with purging
Redundancy techniques have become increasingly popular over the last few years as a tool to decrease latency. For instance, in Vulimiri et al. (2013), the authors, based on
empirical study, argue that redundancy can be effective in reducing latency in a large class of applications. The authors in Joshi, Soljanin, and Wornell (2017) model a cloud
computing set-up as an FJ system with identical servers and study various redundancy techniques thoroughly. Such techniques typically create redundant tasks for each job with the hope of achieving smaller response times because creation of redundant jobs mitigates the synchronisation constraint at the output (see Figure2.1) either entirely (in case of full replication) or partially (in case of partial replication, e.g.,(n, k)Fork-Join in Joshi, Soljanin, and Wornell (2017)). In Poloczek and Ciucu (2016), the authors discuss
replication strategies and compute probability bounds on response times. Based on the bounds, they also devise a replication strategy that improves the stability region of classical FJ systems. In this section, we show that our Markov-additive framework for
a general FJ system developed in Section 4.1 can be immediately applied to study a purging replication strategy in an FJ system.
A purging replication strategy assigns (or replicates) each incoming job to each of the N available servers without splitting. Since there is no division of workload, there is no need to wait for all servers to finish executing their tasks. As such, a job leaves the system as soon as any of its N tasks, which are identical copies of the job itself, is exe- cuted. Therefore, there is no inherent synchronisation constraint at the output. Purging enforces that as soon as the first server executes its task, all other servers immediately discontinue their tasks at that time and take up the task of the next job, if available. The servers are therefore work-conserving. Although it appears to be different from the model described in Section4.1because of the absence of the output synchronisation, it can be viewed as a special case of our model by means of a simple analogy. As done in case of an FJ system with blocking servers in Section 4.2, an FJ system with N het- erogeneous work-conserving servers governed by a purging replication strategy can be viewed as a hypothetical work-conserving system with a single server whose service times are now distributed as ˜Si =D minn∈[N]Sn,i. Therefore, the steady-state waiting time has the following representation
˜ W =D max k∈N0 Zk with Zk:= k
∑
i=1 ZiA, (4.6.1)where ZiA:=minn∈[N]Sn,i−Ai for all i∈N and set Z0:=0. Also define
ρk(s):=k−1log E[exp sZk], and ρ(s):= lim k→∞k
−1log E[exp sZ k].
The upper bound on the tail probabilities of the steady-state waiting times can then be derived directly from Theorem4.1.2. Therefore, we have the following corollary to Theorem4.1.2. The transformed kernel ˜L is calculated using (4.1.5).
Corollary 4.6.1 (Replication with purging). Consider an FJ system with N parallel work-