The project I2C aims at integrating data processing as well as exploitation capabilities in order to detect all types of ships –medium and large– and to identify the threats at sea in order to make a quick report toward the authorities and plan for efficient and relevant actions.
1Integrated system for interoperable sensors & Information sources for common abnormal vessel behaviour detection & Collaborative identification of threat
Sensors Databases
SITU – Maritime Map
Human Operator Multi-Agent
System
Maritime Rule Events
RE
Information
Events
Alerts
Feedbacks
Figure 4.1: Architecture of I2C.
The first objective of the project is the deployment of sensors and radars on several platforms: mobile ones like planes or helicopters for video cameras or immobile ones for High Frequency Surface Wave Radars. All these captors send data to a centralized architecture of I2C. Thus, the first I of I2C stands for Integrated system for interoperable sensors.
Second, I2C retrieves all information coming from different sources in order to aggregate data on the different ships in the monitored area. This data is available on a graphical interface to be used by the users. Besides, using this data and the information, the system is able to detect abnormal behaviours of ships. That is the Information sources for common abnormal vessel behaviour detection part of I2C.
Finally, when abnormal behaviours are detected and alerts triggered, the system aims at proposing a way to identify and understand the underlying threat. This is the Collaborative identification of threat.
In a row, the I2C project proposes a new generation of innovative sea border surveillance end to end systems integrating key existing and in development capacities to track all vessel movements and activities to early identify and report on EUROSUR threats (clandestine immigration, law enforcement, illegal fishing, terrorism...) associated with detected abnormal vessel behaviours.
4.2.1 Architecture of I2C
Figure 4.1 presents the global architecture of the system I2C. One of the main features of I2C is to propose data and information coming from different sources. Indeed, several sensors are deployed on coastal platforms as well as on mobile devices: Long range High-Frequency (HF) radars, High-Frequency Surface-Wave (HFSW) radars, video
Figure 4.2: I2C Common Operational Traffic Picture.
cameras, satellite picture,. . . These devices come with integrated automatic tracking and recognition tools. In addition, information on the different ships can come from other sources like captaincy manifests, web pictures, Automatic Identification System (AIS)2data, . . . Therefore, the system I2C also integrates this data into a knowledge database to be used by the users and the various components of the system.
Once all data is available, the system proposes a visualisation of the monitored area using a cartographic representation, it is called the Common Operational Traffic Picture (COTP). Figure 4.2 illustrates this representation. Using a touch screen interface, the users, the operators involved in the surveillance process are able to display the information on a desired vessel. They are also able to designate an interest on a given vessel and focus on it in order to analyse its behaviour. Thus, the main objective of this component is the fusion of the information coming from the various sources and their display on a graphical interface for the users. There are two kinds of user: the operators monitoring the system and triggering alerts when a ship has a suspicious behaviours; the experts that receive the alerts and are able to identify the related threat or danger, they give a semantic definition of the alert.
Using the COTP, the users are able to identify a ship and see the available information on it: AIS identification, name, color, flag, speed, orientation, . . . These information are sent by the system to the component Behaviour Analysis (BEAN). BEAN is composed of a rule engine and a multi-agent system as shown on figure 4.3.
The rule engine is the entry point of the data. Its purpose is to detect specific anomalies and events happening on the vessels in the monitored area. The rules are designed to detect events that are known to be relevant in the abnormal behaviours cases: unusual events like
2AIS is a radio positioning and vessel identification system in maritime traffic.
Alert Manager
Rule Engine Multi-Agent System
Behaviour ANalysis COTP
Maritime Surveillance Operator events
Data from COTP
alerts
alerts
feedbacks feedbacks
Figure 4.3: Architecture of the BEAN Component.
a stop in open sea for engine failure but also events that are against the maritime legislation, implanted in the rule engine. Rules 4.1 and 4.2 illustrate rules of the rule engine.
if speed > 15 and area = ”Harbour” then Detect(speedAnomaly) (4.1)
if shipType = ”cargo” and speed = 0 then Detect(cargoStopInOpenSea) (4.2) The first rule allows the detection of a ship with a speed above the speed limit in an harbour area, the second one allows to detect the stop of a cargo ship in open sea without apparent reasons.
The detected events are then sent to the multi-agent part of the BEAN component. The MAS role is to combine the events and the anomalies in order to compute a suspicious level for each monitored vessel –see section 4.2.2. The rule engine and the multi-agent system have complementary roles. Indeed, the rule engine performs a first pruning of the data of COTP in order to identify relevant events from irrelevant ones. Then, the multi-agent system use these events and combine them in order to compute the suspicious level of a ship.
The aim of the whole Behaviour Analysis is to receive information and analyse all the events that are happening on the ships in the monitored area. This analysis allows BEAN
to associate a suspicious level to each ship and the component is thus able to detect the anomalies in the ship behaviours. Finally alerts are sent to the operators when it is found to be necessary.
Ultimately, the operators can compare the alerts triggered by BEAN with what they analyse according to their knowledge and experience. If they agree with the automatic alert, they transmit it to the concerned authorities in order to identify and answer the situation (thread to tackle, need for assistance,. . . ). On the contrary, if the operators do not agree with the alert from BEAN, they can send a feedback aiming at the correction of the system. A feedback is also sent when the operators spot an alert that has not been triggered by BEAN.
I2C is a European Project aiming at the surveillance of maritime areas to identify and answer threats and issues. This project integrates acquisition tools as well as anomaly detection mechanisms. In the BEAN component, we now focus on the role of the multi-agent system and how the different parts of MAS4AT can be used for it.
4.2.2 The Role of the Multi-Agent System
In the component Behaviour Analysis (BEAN), the rule engine is combined to the self-adaptive multi-agent system. Indeed, the events and anomalies detected by the rule engine are sent to the MAS in order to be processed by the agents. Then, the MAS decides to send an alert to the operators, if an abnormal behaviour is detected. The role of the multi-agent system is triple.
First it evaluates and represents the behaviours of the ships. Indeed, the MAS should integrate agents to represent each ship in the monitored area. Then, according to the events sent by the rule engine, the agents should be able to represent the behaviours of the ship in the current situation. This behaviour is defined by a series of events that happened on the ship. In MAS4AT, there are entity-agents that can represent the ships. They are also able to compute a value, called Entity Behaviour Value (EBV), that is a numerical representation of a behaviour at a given time and it is based on a combination of events. As this value is given by a function –see 3.2– the agents are also able to give a graphical representation of the behaviour value over time. As the more the EBV is high, the more the entity behaviour is suspicious, this curve represents the evolution of the suspicion level of the entity during time and according to the events from the rule engine.
Second, the agents representing the ships should be able to decide to trigger an alert towards the operators when its behaviours is considered too suspicious. In MAS4AT, the entity-agents are able to compare their EBV to a fixed threshold. This threshold is a representation of the need for an alert when a behaviour is too suspicious. Indeed, when the EBV is superior to the threshold, the corresponding entity-agent is able to trigger an alert onto the related entity. More specifically, each time an event is received by the agent, its values are used to combine it with the previous events. When the combination reach a critical state, an alert is triggered.
Third, the BEAN component has to learn the values of the different possible events.
These values are a numerical representation of the importance of the event. It has been exposed in chapters 2 and 3 that it is not realistic to ask the human operators to give a value to each events because it is their own experience and knowledge that build it. MAS4AT integrates agents to represent the events and their values as well as learning mechanisms to learn the event values. The learning is based on the feedbacks from the operators correcting a wrong or a missing alert.
The multi-agent system in the BEAN component of I2C should be able to represent the entities and the events happening on them. These events have to be valued and combined by the agents and according to the data from the rule engine in order to raise alerts when the entity behaviour is considered too suspicious by the agents. The model of MAS4AT described in chapter 3 is well suited to perform these tasks.