In this chapter, we have introduced an efficient discretization algorithm in PTIME that solves the problem of computing time-bounded reachability probabilities in locally uni-form CTMDPs with respect to time- and history-dependent late schedulers.
To the best of our knowledge, this is the first time that an automatic analysis of time-bounded reachability objectives becomes feasible for time-dependent schedulers. More-over, the main advantage of our approach is that we are able to bound the error that is induced by the approximation algorithm in advance. In particular, the maximal admissi-ble errorε > 0 can be specified a priori.
The computation is done by applying the well-known value iteration algorithm [Ber95]
to the CTMDP’s discretized MDP. We choose the value iteration approach over other methods like LP-solvers, as it has major advantages in our setting: During the value it-eration steps, it is possible to extract the optimal scheduling decisions and to synthesize anε-optimal τ-scheduler whose decisions maximize the reachability objective. Further, the iterative computation allows us to compute time-bounded reachability probabilities incrementally: As a byproduct of the value iteration for a time boundz, we obtain the reachability probabilities for all smaller time boundsz′ <z (where z′is a multiple ofτ) with minimal computational overhead.
Related work. In the literature, the analysis of CTMDPs has received scant attention.
Most of the existing results focus on optimizing criteria such as the expected total re-ward [GHLPR06, Mil68a] or the expected long-run average rere-ward [dA97, GHLPR06, Mil68b]. Directly related to the results of this chapter is the work in [BHKH05], which provides an algorithm that computes time-bounded reachability probabilities in globally uniform CTMDPs. However, its applicability is severely restricted, as global uniformity
— which requires the sojourn times in all states to be identically distributed — is hard to
achieve. We shortly discuss the reason for this:
The approach for the analysis of time-bounded reachability probabilities that is taken in [BHKH05] refers only to time-abstract schedulers, which are strictly less powerful than time-dependent ones [BHKH05, NSK09]. Moreover, as observed in [BHKH05], the uniformization approach that is known from Markov chain theory does not work for CTMDPs and time-abstract scheduler classes: Intuitively, uniformization introduces self loops (or copy states, in case of local uniformization) in the CTMDP model. Thereby uniformization changes the structure of the model. These structural changes expose significant information to history dependent (but time-abstract) schedulers and can be used to estimate the timed behaviour of the system (although the scheduler class is time-abstract). A formal proof of this is included in [BHKH05]. Due to similar reasons, lo-cal uniformization fails for all non-trivial time-abstract scheduler classes as proved in Sec. 4.3 (see page 103).
Recently, maximal reachability probabilities in CTMDPs have been studied in stochas-tic timed games [BF09, BFK+09]: However, the authors of [BFK+09] also consider the strictly weaker classes of time abstract schedulers, while [BF09] addresses the decidabil-ity problem for qualitative reachabildecidabil-ity probabilities in stochastic timed games, that is, reachability probabilities that are 1 or 0, respectively.
Hence, both approaches differ considerably from our results: The time-dependent schedulerML-schedulers that we use are proved to be strictly more expressive (that is, they generally induce strictly higher probability bounds) than the time-abstract sched-ulers that are considered in the related work. To the best of our knowledge, no analysis techniques are known for time-dependent scheduler classes.
Therefore, this chapter extends the existing results considerably: We provide an effi-cient algorithm that computes time-bounded reachability probabilities for the class of time- and history-dependent schedulers up to an a priori given error boundε. Moreover, we relax the restriction to global uniformity in [BHKH05] and allow different states to have different sojourn time distributions.
Chains
It is what I sometimes have called ”the separation of concerns”, which, even if not perfectly possible, is yet the only available technique for effective ordering of one’s thoughts, that I know of.
(Edsger W. Dijkstra) Interactive Markov chains (IMCs) comprise both nondeterministic choices and expo-nentially distributed delays. Hence, in the family of stochastic models they are related to CTMDPs. However, subtle differences exist: Whereas CTMDPs closely entangle nonde-terminism and stochastic behavior in their transition relation, IMCs strictly separate the two aspects and distinguish between Markovian and interactive transitions.
The different approach taken in IMCs is not surprising, given the fact that IMCs orig-inate in stochastic extensions of classical process algebras. As such, they overcome the absence of hierarchical and compositional facilities in purely stochastic dependability models like CTMCs and SPNs [Mol81, Nat80]. Apart from IMCs, many efforts have been undertaken to vanquish this limitation, including formalism like the stochastic Petri box calculus [MVCR08], Statecharts [BHH+09] and in particular, the TIPP [GHR93], PEPA [Hil96] and EMPA [BG98, BG01] process algebras. In this thesis, we focus on IMCs which share most of the other approaches’ benefits while preserving a succinct and accurate semantics.
Since IMCs smoothly extend labeled transition systems (LTSs), the model has received attention in academic and in industrial settings [BCH+08, CGH+08, CHLS09]. In prac-tice however, the theoretical benefits have partly been foiled by the fact that for a long time, the analysis of IMCs was restricted to those instances, where the composed IMC could be transformed into a CTMC.
Beyond these special cases, IMCs also support nondeterminism which arises both im-plicitly from parallel composition and exim-plicitly by the deliberate use of underspecifica-tion in the model [HHK02]. In contrast to CTMC-based models, all of these aspects can neatly be represented in the IMC formalism; therefore, IMCs are strictly more expressive than CTMCs.
The work in [Joh07] is the first approach towards an analysis of nondeterministic IMCs, i.e. of IMCs that cannot be transformed into a CTMC. It relies on a measure preserving transformation from IMCs to CTMDPs and the time-bounded reachability algorithm from [BHKH05]. The latter relies on globally uniform CTMDPs which are obtained by the transformation in [Joh07, BHH+09] if the underlying IMC is also globally uniform, that is, if all Markovian states have the same sojourn time distribution.
Apart from these special cases, no analysis techniques exist for the general setting where IMCs are neither globally uniform nor can they be transformed into an equiva-lent CTMC. In this chapter, we close this gap and provide a model checking algorithm that works for arbitrary IMCs. Our approach extends the discretization technique that is used in Chapter 5: Instead of only consideringtime-bounded reachability objectives, we extend our results to time intervals, that is, we maximize the probability to visit a goal state during a giventime interval. We then use a fixed-point characterization to discretize an IMC and to obtain aninteractive probabilistic chain (IPC) [CHLS09]. Our main contri-bution is the proof that the IPC’s maximum step-interval bounded reachability coincides (up toε) with the maximum time-interval bounded reachability probability in the under-lying IMC. As a final step, we adapt the value iteration algorithm to IPCs and compute the step-interval bounded reachability probabilities.
On the specification side, the continuous stochastic logic (CSL) [ASSB96, BHHK03]
permits to specify a wide variety of performance and dependability measures. It has orig-inally been devised for model checking CTMCs. Therefore, Sec. 6.5 proposes an adap-tation of CSL to IMC which enables us to reason about the maximum and minimum achievable probability for CSL path formulas. We then develop an algorithm to automat-ically model check CSL formulas on arbitrary IMCs.
The crucial point in model checking CSL is the computation of time-interval bounded reachability probabilities. Having achieved the latter, we obtain a model checking algo-rithm which has a worst-case time complexity ofO(∣Φ∣ ⋅ (n2.376+(m + n2) ⋅ (λb)2/ε)), where∣Φ∣ denotes the size of the CSL formula, n, m are the number of states and transi-tions of the IMC, resp., andb and λ are the maximum upper time interval bound in Φ and the IMC’s maximum exit rate, respectively.
As in the previous chapter, we present all results only for maximum time-bounded reachability probabilities. However, all proofs carry over when minimizing the interval-bounded reachability probabilities.
Organization of this chapter. Section 6.1 formally introduces IMCs. In Sec. 6.2 we ob-tain a fixed-point characterizations for time-interval (and step-interval) bounded reach-ability in IMCs (respectively in IPCs). A major contribution are the correctness proofs in Sec. 6.3 which provide the theoretical basis for the value iteration algorithm that we present in Sec. 6.4. Section 6.5 introduces the logic CSL and discusses how the interval bounded reachability analysis can be applied to the model checking problem for CSL on IMCs. Finally, we provide some experimental results obtained by our prototypical
implementation in Sec. 6.6.