Análisis estadístico

This section reports experimental results obtained on the Train-Gate-Controller (section3.2.1), the Alpha algorithm (section5.3.2), the robot foraging scenario (section3.2.1), the autonomous robots example (section3.2.1), and the MSI, MESI, and MOESI cache coherence protocols [BM04]. The experiments were run on an Intel Core i7 CPU 3.4GHz with 8 GB RAM running Linux ker- nel version 3.19.4. The results are reported in Table7.1when verifying said scenarios against the following specifications, respectively.

φT GC = ∀({u,v},{x,y})AG (pt , u) → Kptrainu (¬(pt , v) ∧ ¬(nt , x))
∧

AG ((nt , x) → K_ntrainx (¬(nt , y) ∧ ¬(pt , u))) φAA= ∀{v}KvGF (con, v))

φRFS = ∀({u},{x})AG ((f, x) → KTRu (f, x))

φAR = ∀({v},{x})AG((h_1 , v) → KT R1v (gr_1 , v) ∧ (h_2, x) → KT R2x (gr_2 , x))

φcache= ∀{i,j}AG((modified , i ) → Ki(invalid , j ))

The cutoff detection time is reported in the fourth column, the number of reachable states of the cutoff model is shown in the fifth column, and the overall time required for the verification follows in the last column. For each scenario not reporting a cutoff, the corresponding specifi- cation was satisfied by the abstract model; thus the cutoff identification procedure did not take place. In these cases, the fifth column indicates the number of reachable states of the abstract model.

7.2. Experimental Results 171

#Trains #States Time (s) Memory (KiB)

(1,1) 16 0 8774

(10,10) 1.15 × 1027 3 10005

(20,20) 2.30 × 1013 _{128 47792}

(30,30) 3.57 × 1019 1317 60998

(40,40) _{timeout timeout timeout}

Table 7.2: Verification results for traditional model checking.

From the table, we can observe the time required by the PIIS procedure for the cutoff detec- tion. This almost equals the time required for the whole verification process. This is generally expected since PIIS identifies a cutoff by building the abstract system which is of exponential size in the number of variables encoding the templates. In comparison, the table exemplifies the efficiency of the model checking procedures for the SMR, SGS, and SFE classes. These experimental results confirm advantages following the theoretical results when protocols can be expressed by these classes.

Finally, in comparison with the parameterised model checking techniques introduced in this thesis, Table7.2shows the intractability of the problems here considered in traditional model checking: the time and space requirements grow exponentially in the number of agents to consider. In our case the base model checker MCMAS could not verify the Train-Gate-Controller with 80 trains within the timeout of one hour.

Conclusions

With the deployment of systems based on MAS-architectures there has been a growing interest in their verification. Considerable progress has been made in model checking MAS against specifications based on temporal, epistemic, deontic and strategic properties. Open-source implementations based on efficient symbolic approaches have been put forward and compared. While this work has proven to be valuable, it is limited to scenarios where the number of components is known at design time. This is not a realistic assumption in certain MAS where the number of components cannot be known before deployment. A typical case is robotic swarms whereby the properties of the swarm need to hold irrespective of how many robots are present in the system.

8.1

Summary of thesis contributions

In this thesis we developed methodologies to solve the parameterised model checking problem for MAS in a number of semantical classes. Specifically, we introduced procedures for the cutoff identification of a given unbounded system. When a cutoff can be determined, the parameterised model checking problem can be solved by model checking all system instances up to the cutoff. We showed the procedures to be sound.

Table8.1summarises the theoretical results obtained. 172

8.1. Summary of thesis contributions 173

Semantics Cutoff Soundness Completeness #Systems to check

PIS _{S(m, td(∀}˙ _{vφ(v) + 1)) ˙≤sS(c) Yes Yes N/A}

PIIS S(m) ≤ˆ gsS(c) Yes No N/A

SMR (max(1, m.i + |Di|))1≤i≤|T | Yes No Q1≤i≤k(c(i) − m(i) + 1)

SGS (max(1, m.i)1≤i≤|T | Yes No 1

SFE (max(1, m.i))1≤i≤|T | Yes Yes 1

Table 8.1: Summary of theoretical results.

In chapter 4 we presented a sound and complete technique for parameterised interpreted systems. The cutoff detection procedure was based on the identification of a concrete system that can simulate the pruned computation forest of the abstract model. Soundness of the technique was assessed by means of cycle-stuttering simulations preserving the satisfaction of formulae up to a level of temporal depth. Upper bounds on the size of cutoffs were given from which the completeness of the procedure followed. The applicability of the procedure was theoretically showcased on the Beta swarm aggregation algorithm.

In chapter 5 we analysed a sound but incomplete technique for parameterised interleaved interpreted systems. Since the parameterised model checking problem for PIIS is in general undecidable, no complete results can be established in general. The cutoff detection procedure put forward relied on the existence of a gs-simulation between a concrete system and the abstract system. Intuitively, the condition of a gs-simulation expresses that all the agents can always succeed to globally synchronise. The application of the procedure on the Alpha algorithm and on cache coherence protocols illustrated that concrete scenarios often adhere to this condition.

Given the above procedure is in exponential space, in chapter6we identified the SMR, SGS, and SFE classes of PIIS for which we devised polynomial cutoff detection techniques. In sec- tion6.2we presented an incomplete technique for the SMR class. In section6.3we analysed an incomplete technique for the SGS class. In section 6.4 we studied a complete technique for the SFE class. Incompleteness for the SMR and the SGS classes follows in the absence of an agent-environment simulation between the templates, since in this case the techniques cannot assess the correctness of a given specification. By contrast, we can always assess the correctness of a specification on SFE systems. This level of confidence, which follows from the decidability result of Corollary6.4, comes with considerable limitations to the range of sys-

tems the technique can be applied to. For instance, the result cannot be applied to any scenario where the agents evolve in any other way other than lock-step evolution. Other systems may be modelled as SMR or SGS systems. SMR is suitable for scenarios requiring refined interactions between agents of different roles, whereas SGS is suitable for simulating synchronous seman- tics. In general, the technique for the former class generally returns larger cutoffs than those for the latter class. Both techniques are limited by the requirement of an agent-environment simulation between the agent and environment templates. This makes it difficult to model certain applications of interest, such as cache coherence protocols.

The experiments obtained on the experimental model checker MCMAS-P that we introduced in chapter7confirm the correctness and attractiveness of the approach taken in this thesis.