Resumen

At a fundamental level, each agent internalises a performance measure that enables it to

“measure” the expected outcome of a decision. Based on the utility value obtained, an agent can determine whether to actuate the chosen action. This decision-driven

deliberation process occurs at both microscopic and macroscopic levels within a MAS. Each agent is designed to attempt to reach its local utility, while the objective of the MAS is to arrive at a global utility.

As mentioned in Chapter 3, agents can potentially make perfectly rational decisions based on their deliberation process, which may impact or suppress the utility of other agents within the same environment. This is further complicated if multiple agent types are utilised. Often, agent simulations are performed to model how different agent types can

6 A communications approach or protocol whereby one agent can communicate and share information simultaneously with multiple other agents.

42 | P a g e potentially influence one another (Harland, Morley, Thangarajah & Yorke-Smith, 2017;

Logan, 2017; Macal & North, 2014; Weiss, 2013). This is particularly useful in a dynamic environment, as there might be unforeseen environmental variables that can potentially cause processes and expected outcomes to differ (Logan, 2017; Luke, Cioffi-Revilla, Panait &

Sullivan, 2004; Richmond, Walker, Coakley & Romano, 2010). This phenomenon is known as an agent-driven side-effect within a MAS.

In order to understand the intention of an agent, i.e. to establish an inter-agent

understanding of how the internalisation of a performance measure affects decisions and outcomes, it is important to establish an understanding of the internal structures of agents, how they operate within an environment, and how their deliberation processes affect the way in which goals are achieved. For the CESIMAS model, this thesis looked at the

mentioned aspects from a MAS perspective only.

At a fundamental level, each agent consists of various attributes and methods that describe facts, information and actions which can be associated with an agent. These characteristics enable an agent to perceive its environment, capture details of the environment’s state, deliberate on a course of action, and effect change in the environment through its actuators (Bear & Rand, 2016; Bleiweiss, 2009; Weiss, 2013; Wooldridge, 2008). The actions that might be taken by an agent should provide the best possible utility but should not prevent/suppress other agents from striving to do the same.

Figures 3.2 and 3.3 show different agents interacting in an environment, and an agent’s representation respectively. Figure 3.2 illustrates agent interaction within an environment.

The visualised environment creates a two-dimensional space whereby an agent can interact with its neighbouring agents, also referred to as a “Moore Neighbourhood”⁷. Figure 3.3 depicts the internalised state of an agent. Each agent has a set of attributes and methods which it can utilise to make decisions, interact with other agents, and convey information.

7 In a two-dimensional Moore Neighbourhood, a 3-by-3 grid is created virtually, and the agent is placed in the centre of the grid. The agent can then only interact with the surrounding eight agents in all the blocks adjacent to the block in which the agent has been placed.

43 | P a g e Figure 3.2: Different Agents Interacting in an Environment (Adapted from Macal & North,

2010)

Figure 3.3: An Agent's Internal Representation (Macal & North, 2010) Within a MAS, it would be difficult to ensure that agents do not suppress others unintentionally, especially in a dynamic environment such as CIIP. Every agent has a

prioritised set of goals that need to be addressed, which can be modelled by making use of

44 | P a g e the Belief-Desire-Intention (BDI) model (Bear & Rand, 2016; Ebrahimi, 2018; Georgeff, 2012;

Horvitz, 2013, Howard & Cambria, 2013; Johnson et al., 2017; Macal & North, 2010). The BDI model as depicted in Figure 3.4, provides a platform whereby agent-driven deliberation and intention management can be modelled and visualised.

As per Figure 3.4, each agent has a belief that describes the agent’s understanding of the information obtained from its percepts. The Belief-state is then augmented with the Desire component, which describes the “eagerness” of an agent to reach the best possible utility (Chen, 2015; Nunes & Luck, 2014; Russell, 2016; Wooldridge, 1999). These two aspects then contribute towards an agent’s Intention, describing what course of the action the agent wants to execute within the environment, and what the expected outcome will be

(Ebrahimi, 2018; Padgham, Scerri, Jayatilleke & Hickmott, 2011; Russell, 2016; Wooldridge, 1999).

Taking this approach enables agents within a MAS to go through a deliberation process, whereby a suitable action can be chosen from a list of actions that the agent is instilled with, instead of having a one-to-one utility-driven function where every percept has a predefined action that “must” be executed (Hegde & Singh, 2013; McHugh et al., 2016; Russell, 2016;

Wooldridge, 2008). The BDI-model enables agents within a MAS-environment to request additional sensory-driven inputs post the agent reaching its Intention-state. This enables agents to determine the best course of action that yields the best possible local and global utility.

Implementing a BDI-driven approach within a dynamic environment such as CIIP can be a difficult task. CII consistently changes as a result of the environment dynamism, and this results in agents having to make decisions with inconsistent information. State changes can occur while an agent is performing its deliberation processes, making it difficult for an agent to effectively gauge the correct Intention state. Implementing cognitive capabilities can potentially address this caveat. These capabilities are discussed in detail in Chapter 5.

Figure 3.4 illustrates the Belief-Desire-Intention model.

45 | P a g e Figure 3.4: The Belief-Desire-Intention Model (Wooldridge, 1999)

Section 3.1 through section 3.4 was dedicated towards introducing MASs, the dynamics of such systems, how information is exchanged, and how decision-making occurs within such models. MASs contain characteristics which could be desirable when applied to the problem domain of CIIP. Section 3.5 concludes the chapter by providing a brief review.

3.5 Conclusion

Chapter 3 was dedicated to defining agent technology and the high-level applications thereof to the problem domain of CIIP. In Chapter 3, agent technology was defined within the context of this thesis.

This was followed by a discussion revolving around MASs and the use thereof to perform concurrent and simultaneous problem-solving capabilities in any environment. MASs hold some ideal characteristics that were useful for the CESIMAS model.

Chapter 3 then elaborated on the dynamism which occurs in an environment where multiple agents and multiple agent types operate. When multiple agents operate in a dynamic environment, the possibility exists that they can suppress other agents, which can be an unforeseen caveat in a MAS environment. This caveat can result in agent-level local maximum utilities being achievable, but will inevitably reduce the potential maximum global utility of the MAS as a whole.

To address this potential caveat, a section was dedicated to discussing agent-driven

intentions. As part of this process, agents can effectively perform deliberation (similar to the

46 | P a g e discussed BDI model), and given a set of percepts, can assess which course of action will result in the highest state of utility.

Chapter 3 outlined four research questions which served as the general outline:

Research Questions

RQ 3.1 What is an agent and which characteristics are ideal for the problem domain of CIIP?

RQ 3.2 Why can agent-technology be used to solve complex problems of a dynamic nature?

RQ 3.3 What is a Multi Agent System and how can it be applied to the problem domain of CIIP?

RQ 3.4 How will MASs contribute towards the CESIMAS model?

RQ 3.1: What is an agent and which agent characteristics are ideal for the problem domain of CIIP?

Section 3.1 addressed the definition of an agent within the context of this thesis and the problem domain that it addresses. On a very high level, an agent can be seen as an autonomous entity that can perceive its environment and perform actions in that environment based on its actuators.

An agent can be developed to perform actions autonomously, by instilling the agent with a set of characteristics and abilities that allow it to perceive its environment, deliberate on the state of the environment, determine a suitable action based on its performance element, and then perform those actions.

Agent technology can be useful in the problem domain, as agents can be autonomous by nature, perform introspection when given a set of characteristics, analyse and deliberate based on sensory information obtained from percepts, and can effect or execute actions based on its analysis and understanding. This is some of the ideal characteristics which can be applied within a CIIP environment, as ensuring effective CIIP can be a resource-intensive task, specifically where a human-resource is involved.

RQ 3.2: Why can agent-technology be used to solve complex problems of a dynamic nature?

The answer to this question has some overlap with aspects addressed by RQ 3.1. Utility-driven agents can possess the ability to effectively and efficiently manage their deliberation and intention management processes.

Agent-technology consists of some of the ideal characteristics that enable automated decision-making and action execution to occur without the need for human intervention.

Although this key feature can have both a positive and negative impact on a dynamic environment, in can be more beneficial than detrimental to the problem domain of CIIP.

47 | P a g e Agent-technology provides a platform for knowledge sharing and learning to occur on a continual basis. Any mechanisms or technology that operates in a dynamic environment where the only constant element is change, needs to adapt over time to ensure that it can maintain the expected level of utility.

RQ 3.3: What is a Multi Agent System and how can it be applied to the problem domain of CIIP?

Section 3.2 addressed MASs within the context of this thesis and its problem domain. A Multi Agent System consists of multiple agents (and potentially multiple agent types), all operating in an environment both simultaneously and concurrently.

MASs provides the ability to deploy multiple agents to solve a problem in a team-like manner, providing a more efficient approach to the problem-solving process. This is especially useful for the problem domain of CIIP, as the CII is by nature dispersed, continually undergoes changes in its state, and is open to exploitation from multiple sources.

The consistent changing of elements in the CII environment makes it difficult to ensure effective and efficient CIIP. This holds especially true for the more conventional CIIP mechanisms which were discussed in Chapter 2.

MASs provide the ability to do the following within the context of CIIP:

• Share knowledge between agents – improving utility;

• Communicate sensory information to improve the deliberation and decision-making processes;

• Effective and efficient decision making via multiple agents being able to analyse potential problems and provide their “contextualisation” thereof;

• Cooperation – the ability to work with other agents to address potential issues in a distributed manner;

• Distributed Control – removing the single point of failure from the protection mechanism or technology; and

• Adaptive capability – to change to events in the dynamic environment as and when they occur.

RQ 3.4: How did MASs contribute towards the CESIMAS model?

MASs provided the underlying architectural analogy utilised in the CESIMAS model. A MAS-approach provided the ideal foundation for the CESIMAS model, given its characteristics, processes, and benefits, which were discussed in Chapter 3.

48 | P a g e MASs concepts, processes, and distributed nature played a pivotal role in the development of the CESIMAS model. MASs were augmented with AISs, (discussed in Chapter 4), and enriched with self-awareness and AmI capabilities (discussed in Chapter 5).

The CESIMAS model combines desirable characteristics from the research fields mentioned in the preceding paragraph. Combining different research fields can potentially lead to caveats, and these are assessed and discussed starting in Chapter 6.

49 | P a g e Analogy is a wonderful, useful and most important form of thinking, and biology is saturated with it… In its central content, biology is not accurate thinking, but accurate observation and

imaginative thinking, with great sweeping generalisations. – Anthony Standen

50 | P a g e