To decide which agent language/framework to use we drew up a list of requirements to help identify which language would best suit our needs. The list of requirements we generated are as follows:
1. allow for high level descriptions of agent activities with minimal coding
2. can show a distinct time line of events to demonstrate when activities occur
3. an embedded geographic model which requires no external code or environments
4. allow agents to perform human specific behaviour, such as:
(a) thinking about problems
(b) reasoning about the implications of beliefs and causality of actions (c) off task behaviour and multi-tasking
(d) making mistakes
(e) communicating with each other
(f) taking varied amounts of time to complete tasks
When considering these requirements it became clear that Brahms was the only language we could find to match these requirements. Brahms has the high level ab- stractions of activities all contained within a time line of events. It has a simple geographic syntax for describing the possible locations, the distance between them, their associations with each other and it is very simple to tell agents to move from location to location. Brahms was also developed for simulating humans, so it meets our human behaviour requirements. Brahms has also been used at NASA to simulate human behaviour for over 10 years.
Other agent languages considered were AgentSpeak(L) and Jason, 3APL, and Goal. These are all common and popular languages used to program agents, which is their primary function. Brahms on the other hand was developed to model both humans and machines at NASA [76, 73, 21]. Although representation of humans may be possible in all the above languages, it is only Brahms which has been specifically designed and used to model human behaviour. Brahms achieves this representation of humans by modelling the objects they use, the environment they are in, their thought and work processes, communication, and other concepts where humans and agents will typically collaborate. AgentSpeak(L) and Jason, 3APL, and Goal are focussed on the develop- ment and behaviour of the agents, including competitiveness and lack a framework for shared achievement [11]. However, it could be argued that any of these languages could be used for modelling humans, even though they have not been specifically designed to do so. This then leaves the issue of defending how accurate this representation of a human is, whereas with Brahms there are already papers and experiments demon- strating its use in modelling humans [76, 73, 21] and thereby we can safely assume it meets both requirements 4 and 5. Brahms also allows for a high level of abstraction, where capabilities of agents can be assumed. This high level of abstraction, which matches requirement 1, allows for easier modelling of humans because we can assume they can perform certain tasks, such as moving a simple object, without any difficul- ties. Whereas an agent centred approach like 3APL, GOAL and AgentSpeak(L) do not usually allow for such high level of abstraction, they require us to model every action. Activities in Brahms are inherently linked with time, meaning no extra coding is required which works towards meeting requirement 2. The other agent languages are more interested in the events themselves than the time they take to complete, although it is possible for them to represent this time it will require additional coding making the simulation more verbose. Brahms is also the only language which matches require- ment 3; an embedded graphical model. AgentSpeak(L), 3APL and GOAL all require the geographic models to be described in another language such as Java or C, making the simulations much more verbose and add an additional language to the verification process. Brahms’ graphical model is simple to program, has very little extra syntax and allows easy reference to these locations in the simulations.
In summary, we are interested in analysing the teamwork aspect of human-agent teams, examining the interactions and work processes the teams use to complete their task. Therefore it only seems natural that we choose a framework designed to model humans and work processes, such as Brahms. Brahms is also the only language we could find that meets all of our requirements easily, making it an obvious choice. Additionally Brahms has already been used to model human-agent-robot teamwork, meaning there are existing models/examples already available for us to apply verification.
Chapter 4
Formal Verification - Techniques
and Applications
The primary concern of this thesis is to perform formal verification of models of human- agent teamwork. In this chapter we discuss: what we mean by the term formal verifica- tion; why we wish to perform agent verification; and which formal verification technique we use and why. The structure of this chapter is as follows:
• Formal Verification; here we explain what we mean by formal verification and formal operational semantics
• Model Checking; here we explain the formal verification technique model checking and the model checking tool Spin
• Other Formal Verification Techniques; here we briefly explain opposing verifica- tion techniques to model checking
• Agent Verification; here we explain how agent verification has previously per- formed on agent based systems
• Verification of Agent Languages; here we explain how verification has been per- formed on agent languages
4.1
Formal Verification
Formal verification represents a family of techniques aimed at assessing the correctness of a system design. These techniques have become very popular in hardware design since they can ensure 100% functional correctness of circuit designs [30]. For example, Kaivola and Narasimhan [46] describe the process they used to verify the floating-point multiplier in the Intel IA-32 Pentium microprocessor. Formal verification is always performed against a set of requirements, i.e., a specification. A formal specification is a concise mathematical description of the behaviour and properties of a system, stating what actions a system can take and what should (or should not) happen [51]. Informal specifications are inadequate for formal verification as they tend to be vague, improper, incomplete, hard to analyse and ambiguous. A multitude of formal languages and logics have been created in order to express as broad a range of properties as possible.
liveness, safety and fairness are typical properties to check. The liveness concept states the system must at some point perform this action, e.g., “the spacecraft will
take off”. Safety is a concept which states something must never happen, e.g., “while performing manoeuvres in space, no doors can be opened”. Fairness is a concept used when multiple agents are employed; it states that each agent will fairly get the chance to perform an operation with no agent indefinitely occupying the resource. Formal languages may also need to be able to express properties concerning real-time dynamic systems, probabilistic systems and goal driven systems [11].
Essentially formal verification is a reachability test, testing whether a certain state can be achieved which does not satisfy the specification. Formal verification can how- ever be used to identify other faults in a system which are not part of the specification such as deadlock, livelock, race conditions and termination. Deadlock occurs for in- stance when a process will not release a shared resource that other processes are waiting to access and cannot progress any further until they access the resource. Livelock is similar to deadlock, such that no progress is made but no blocking occurs [63], e.g., one processor constantly flips a Boolean to true and in response another flips it back to false. There is no strict definitive definition of a race condition, however race con- ditions generally occur when different processes share a data source without explicit synchronization [60]. Termination analysis is simply a check to identify whether the program will always terminate, such as identifying any infinite loops. Termination is a difficult problem, also known as the halting-problem, and has been the subject to intensive research, for example, in [53, 33, 26].
The most popular approach to formal verification is model checking [27]. Model checking (see Chapter 4.2) creates a model of every single state achievable within a system, the transitions between these states, and also indicates which states are the possible initial states. Every single run (a sequence of state transitions from an initial point to an end point) is checked to ascertain whether or not a formal property holds. Model checking requires a finite model of the system, which we generate from the operational semantics, and the representation of a property to check in some logic.