Benchmarking

We now turn to the time-unbounded reachability problem, i.e., we are interested in the probability to reach a state in G, while there are no constraints on the time to reach the G-states. Let Pr(s, x, ♦ G) denote the reachability probability from state s at time x to reach G within time interval [0, ∞). Using Proposition 3.4, we can characterize the time-unbounded reachability probability as follows

Pr(s, x, ♦ G) =





 Z ∞

0

X

s^′∈S

R_s,s^′(x+τ )e^{−R0τEs(x+v)dv}· Pr(s^′, x+τ, ♦G) dτ, if s /∈ G (3.20)

1, if s ∈ G. (3.21)

The case s ∈ G is derived from (3.6), where the probability to delay in an F -state for zero units of time is 1 and the probability to leave (i.e. taking a Markovian jump) an G-state in zero units of time is 0. When s /∈ G, Eq. (3.20) is similar to (3.7) except that there is no bound on the time to leave a state s /∈ G. Note that in contrast to the time-bounded case, in general it is not possible to reduce the above system of integral equations to a system of ODEs since it has no unique solution.

Solving a system of integral equations is generally time consuming and numeri-cally instable, we propose to investigate some special cases (subsets of ICTMCs), for which the reduction to ODEs is possible. Here we consider two such classes, i.e. eventually periodic ICTMCs and eventually uniform ICTMCs. Their com-mon feature is that rate functions of the given ICTMC exhibit regular behaviors after some time T . This allows for computing time-unbounded reachability prob-abilities efficiently (e.g., via DTMCs). In these cases, the problem turns out to be reducible to computing the time-bounded reachability probabilities with time bound T , which has been tackled in the previous section, and determining

reach-Ts,s^′

ns,s^′·P

0 t Rs,s^′(t)

Ts t

0 R_s,s^′(t)

Figure 3.2: Eventually periodic assumption (left) and eventually stable assump-tion (right)

ability probabilities for DTMCs. Both these sub-problems, fortunately, enjoy efficient computational methods.

Eventually periodic assumption. We consider eventually periodic ICTMCs.

Definition 3.7 (Eventually periodic assumption (EPA)) An ICTMC C is eventually periodic if there exists some time P ∈ R>0 such that for any two states s, s^′ ∈ S, there exists some time T_s,s^′ ∈ R_>0 and ns,s^′ ∈ N such that for all t ≥ Ts,s^′:

R_s,s^′(t) = R_s,s^′(t + n_s,s^′·P ).

An example rate function satisfying the EPA is illustrated in Fig.3.2(left). Af-ter time point Ts,s^′, the function Rs,s^′(t) becomes periodic with the period ns,s^′·P , where P is the “common factor” of all the periods of rate functions R_s,s^′(t), for all s, s^′ ∈ S. For any ICTMC C satisfying EPA, let TEP = maxs,s^′∈STs,s^′ and PEP = (gcd_s,s′∈Sns,s^′)·P . Intuitively, TEP is the time since from which all rate functions are periodic and PEP is the period of all the periodic rate functions.

For instance, suppose Rs1,s2(t) = 2 + cos(¹₂t) and Rs2,s3(t) = 3 − sin(¹₃t), and let Ts1,s2 = 10, Ts2,s3 = 15. Then TEP= max{10, 15}=15, P =π, ns1,s2=4 and n_s2,s3=6, and P_EP= gcd{4, 6}·π=12π.

We will show that time-unbounded reachability probabilities for an ICTMC C under the EPA can be computed according to Alg.1and justified by Theorem3.8 (see below). Let us explain the idea in more detail. Due to (3.21), once F states are reached, it is irrelevant how the paths continue. This justifies the model

transformation from C to C[F ]. The reachability problem can be divided into two subproblems: (I) first compute the probability to reach state s^′ ∈ S at exactly time TEP (the second term in (3.22), see below); and (II) then to compute the time-unbounded reachability from s^′ ∈ S to F (the third term in (3.22)).

Theorem 3.8 Let C = (S, AP, L, α, R(t)) be an ICTMC satisfying EPA with According to the Markovian property, it is not difficult to see that

Prs,0(Λs,{s^′_i}06i6n+1) =

Note that here P^D_s′^{C[F ]}

i,s^′_i+1 is the (one-step) transition probability of DTMC D_{C[F ]} from state s^′_i to state s^′_i+1. we obtain, from (3.23) and (3.24), that

Pr^C(s, 0, ♦ F )=X

s^′∈S

Pr^{C[F ]}(s, 0, ♦^=TEPs^′)·Pr^{DC[F ]}(s^′, ♦ F ).

 In the following we will focus on (II): Recall that Pr(s, T_EP, ♦^=PEPs^′) is the probability to reach s^′ from s after time PEP starting from time point TEP in C.

Since after time TEP all rate functions are periodic with period PEP, it holds that Pr(s, TEP, ♦^=PEPs^′) = Pr(s, TEP+ n · PEP, ♦^=PEPs^′),

for all n ∈ N. It then suffices to compute Pr(s, T_EP, ♦^=PEPs^′) for any s, s^′ ∈ S.

This is done as follows, given the ICTMC C with state space S, we build a

DTMC DC = (S, P) with Ps,s^′= Pr^C(s, TEP, ♦^=PEPs^′). Intuitively, Ps,s^′ is the one-step probability (one-step here means one period) to move from s to s^′, and the problem (II) is now reduced to computing the reachability probability from s to F -states in arbitrarily many steps (since the time-unbounded case is considered), i.e., Pr^{DC[F ]}(s, ♦ F ). This can be done by standard methods, e.g., value iteration or solving a system of linear equations, see, among others, [BK08]

(Ch. 10).

Remark 3.9 To obtain P r_EP^C (s, x, ♦ F ), where the starting time is x, we define an ICTMC C^′ = (S, AP, L, α, R^′(t)) such that R^′(t) = R(t + x) and it follows that C^′ still satisfies EPA (with T_EP^′ = TEP − x if x 6 TEP and 0 otherwise;

P_EP^′ = PEP) and Pr^C_EP(s, x, ♦ F ) = Pr^C_EP^′ (s, 0, ♦ F ).

Algorithm 1 Time-unbounded reachability for ICTMCs satisfying EPA Require: ICTMC C = (S, AP, L, α, R(t)), EPA time TEP, period P_EP

Ensure: Pr^C_EP(s, 0, ♦ F )

1: For any two states s, s^′ ∈ S in C[F ], compute the time-bounded reachability proba-bility with time bound T_EP, starting from time point 0, i.e. Pr^{C[F ]}(s, 0, ♦^=TEPs^′);

2: For any two states s, s^′ ∈ S in C[F ], compute the time-bounded reachability probabil-ity with time bound P_EP, starting from time point T_EP, i.e. Pr^{C[F ]}(s, T_EP, ♦^=PEPs^′);

3: Construct a discrete-time Markov chain (DTMC for short) D_{C[F ]} = (S, P) with P_s,s^′ = Pr^{C[F ]}(s, TEP, ♦^=PEPs^′). We denote the reachability probability from s to F in DC by Pr^{DC[F ]}(s, ♦ F );

4: Return P

s^′∈SPr^{C[F ]}(s, 0, ♦^=TEPs^′) · Pr^{DC[F ]}(s^′, ♦ F ).

Eventually uniform assumption. The previous section has discussed rate functions enjoying a periodic behavior. A different class of rate functions are those which increase or decrease uniformly, e.g., an ICTMC in which all rates are a multiplicative of the Weibull failure rate which is characterized by the function f (t) = _α^γ _α^t
γ−1

, where γ ∈ R_>0 and α ∈ R_>0 are the shape and scale parameters of the Weibull distribution, respectively. These distributions can e.g., characterize normal distributions, and are frequently used in reliability analysis.

This suggests to investigate eventually uniform ICTMCs.

Definition 3.10 (Eventually uniform assumption (EUA)) An ICTMC C is eventually uniform if there exists T_EU ∈ R_>0 and an integrable function f (t) : R_>0 → R_>0 such that lim_t→∞Rt

TEUf (τ )dτ → ∞ and for any two states s, s^′ ∈ S and t > TEU, Rs,s^′(t) = f (t) · R^c_s,s′, where R^c_s,s′ is a constant.

In terms of the infinitesimal generator Q(t) of the ICTMC C, EUA intuitively entails that there exists some function f (t) and constant infinitesimal generator Q^c = R^c − E^c (R^c and E^c are the constant rate matrix and exit rate matrix, respectively) such that Q(t) = f (t)·Q^c for all t > T_EU. We also define the constant transition probability matrix P^c such that P^c_s,s′ = ^R

c s,s′

E^c

s .

By restricting to the EUA, one can reduce the time-unbounded reachability problem for an ICTMC C to computing the time-bounded reachability proba-bility with time bound TEU and the reachability probability in a DTMC D_EU^C [F ] with transition probability matrix P^c[F ], where P^c[F ]s,s^′ = P^c_s,s′ for s /∈ F ; P^c[F ]s,s = 1 and P^c[F ]s,s^′ = 0, for s ∈ F and s^′ 6= s. This is shown by the following theorem.

Theorem 3.11 Let C = (S, AP, L, α, R(t)) be an ICTMC satisfying EUA with s ∈ S and F ⊆ S. For TEU ∈ R_>0 it holds that

Pr^C(s, 0, ♦ F ) =X

s^′∈S

Pr^{C[F ]}(s, 0, ♦^=TEUs^′) · Pr^{DCEU[F ]}(s^′, ♦ F ), (3.25)

where D_EU^C [F ] is the DTMC with transition probability matrix P^c[F ].

Proof: Recall that the time-unbounded reachability in ICTMC C can be char-acterized by (3.20) and (3.21). We now write them into a matrix form:

~p(x) = Z ∞

0

M(x, τ )~p(x+τ ) dτ + ~1F, (3.26) where pi(x) is the probability to reach the set of states F starting from state si

and time x; and ~1F is defined as: if si ∈ F then ~1⁽ⁱ⁾_F = 1 otherwise ~1⁽ⁱ⁾_F = 0.

We now show that ~p(TEU), the time-unbounded reachability starting from time TEU in C, can be characterized by the least solution of the following system of linear equations

~ˆp = ^P^c[F ] · ~ˆp + ~1F, (3.27) where ˆpi is the probability to reach the set of states F from state si in a DTMC D^C_EU[F ] with transition probability matrix P^c[F ], i.e., ˆpi = Pr^{DEUC [F ]}(si, ♦ F ).

Note that ^P^c[F ] is P^c[F ] except that ^P^c[F ]_s,s = 0 for s ∈ F (while P^c[F ]_s,s = 1).

For this purpose, we apply induction on the number of steps to reach a state F . We write ~p⁽ⁿ⁾(x) and ~ˆp⁽ⁿ⁾ as the probability to reach the set of states F in n-steps in ICTMC C and DTMC D^C_EU, respectively.

• Base case: By the definition of ~p and ~ˆp, ~p⁽⁰⁾(t) = ~ˆp⁽⁰⁾ = ~1F for any t > TEU. (Note that the probability to reach F in zero steps from a state s /∈ F is 0.)

• Induction step: The I.H., is ~p⁽ⁿ⁾(t) = ~ˆp⁽ⁿ⁾ for any t > TEU. We need to show that ~p⁽ⁿ⁺¹⁾(t) = ~ˆp⁽ⁿ⁺¹⁾ for any t > T_EU. By Eq. (3.26) we have that for t > 0,

~p⁽ⁿ⁺¹⁾(t + TEU) = Z ∞

0

M(t + TEU, τ )~p⁽ⁿ⁾(t + TEU+ τ )dτ + ~1F.

From the I.H. we obtain that

~p⁽ⁿ⁺¹⁾(t + T_EU) = Z ∞

0

M(t + T_EU, τ )~ˆp⁽ⁿ⁾dτ + ~1_F. (3.28) As C satisfies EUA, we have that

M(t + TEU, τ ) = R(t + TEU+ τ ) · D(t + TEU, τ )

= f (t + T_EU+ τ ) · R^c · D(t + T_EU, τ )

= f (t + TEU+ τ ) · ^P^c[F ] · E^c· D(t + TEU, τ ). (3.29) Substituting (3.29) into (3.28), yields

~p⁽ⁿ⁺¹⁾(t + TEU) = ^P^c[F ] · Z ∞

0

f (t + TEU+ τ ) · E^c· D(t + T_EU, τ )dτ · ~ˆp⁽ⁿ⁾+~1F. Since lim_t→∞Rt

0 E_s(τ + T )dτ → ∞, we have 1 − e^{−R0∞Es(τ +T )dτ} = 1. Thus, the probability to leave s in interval [0, ∞) starting at time T is 1. On the other hand, it follows that the above integral represents the probability to leave any state s ∈ S in an interval of time [0, ∞) starting at time t + TEU. Hence we obtain

~p⁽ⁿ⁺¹⁾(t + TEU) = ^P^c[F ] · ~ˆp⁽ⁿ⁾+ ~1F = ~ˆp⁽ⁿ⁺¹⁾.

Since the matrix ^P^c[F ] is stochastic, by the Perron-Frobenius theorem, we have that limn→∞~p⁽ⁿ⁾(t + TEU) and limn→∞~ˆp⁽ⁿ⁾ exist and are equal. We conclude that for any t > 0,

~p(t + TEU) = ~ˆp.

It follows that

Pr^C(s, 0, ♦ F ) = X

s^′∈S

Pr^{C[F ]}(s, 0, ♦^=TEUs^′) · Pr^{C[F ]}(s^′, TEU, ♦ F )

= X

s^′∈S

Pr^{C[F ]}(s, 0, ♦^=TEUs^′) · ~p_s^′(T_EU)

= X

s^′∈S

Pr^{C[F ]}(s, 0, ♦^=TEUs^′) · ~ˆps^′

= X

s^′∈S

Pr^{C[F ]}(s, 0, ♦^=TEUs^′) · Pr^{DCEU[F ]}(s^′, ♦ F ).

This completes the proof. 

Remark 3.12 We note that the two assumptions, EUA and EPA are incompa-rable. There are rate functions (e.g. polynomials) which can not be represented as periodic functions but satisfy EUA; on the other hand, in case of EPA one can, for instance, assign the same sort of rate functions (e.g. sin) with different periods, and thus obtain an ICTMC which invalidates EUA.

In document Propuesta de prototipo para el incremento de la competitividad en Cocostin SA de CV (página 45-58)