A thesis submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (major: computer science) at the University of Girona. It allows agents to schedule their tasks taking into account temporary tasks preferences of the agents.
Overview
This algorithm uses combinatorial auctions to allocate tasks between agents and has some features such as: it considers the capabilities of the task and the agent in the bidding configuration and has a readjustment phase that allows all agents to be allocated in each round of the auction; . A mechanism that allows agents to reschedule tasks when they encounter problems that prevent them from following the initial plan.
Motivation
The RoboCup Rescue simulation environment has all the complex characteristics previously mentioned and is therefore a complex test bed for cooperative multi-agent systems. In this sense, this thesis discusses the use of new technologies based on scheduling and task allocation to reduce damage caused by natural disasters.
Objectives of the research
Thesis outline
The DAI field is divided into two research lines: Distributed Problem Solving (DPS) and research in multi-agent systems. In multi-agent systems, the components are called agents and have the same properties as in real communities where agents must cooperate to achieve their goals and the goals of the communities involved [51].
What is an intelligent agent?
These environments are dynamic in that they change rapidly; this means agents cannot assume that the environment will remain static while they are trying to achieve a goal. Finally, these environments are unreliable in that the action an agent may perform may fail for reasons beyond an agent's control.
Agent environments
This distinction is also used for multi-agent systems, but in that case the differences are obviously only in the software implementation of the agents' behavior [28]. The agents are able to act in an environment; different agents have different “spheres of influence”, in the sense that they will have control over – or at least be able to influence – different parts of the environment.
Goal-oriented agents
Multi-agent cooperation
Task sharing
Information sharing
Handling inconsistency
Multi-agent coordination
In this sense, commitments and conventions play an important role in the agent interaction. Coordination through mutual modeling: This approach is closely related to the model of coordination through joint intentions discussed above.
Task oriented domain
For any given set of tasks X ⊆ T, c(X) is the cost of executing all tasks inX by any single agent. Any given set of tasks has a cost, which may or may not be independent of the agent or agents performing it (this depends on the particular domain or application).
Resource allocation
A TOD specifies the set of all possible tasks that can be executed, a set of agents, and a cost function. By adding more tasks, the cost will remain the same or increase (generally, it will increase).
Auctions
Classifying auctions
First, the one shot auction, which offers a single round, after which the auctioneer awards the item to the winner. Second, the ascending auction in which the price starts low and successive bids are for increasingly large amounts and third descending auction in which the auctioneer must start with a high value and lower the price in successive rounds.
Auction for single items
Order in which the items are auctioned: There are two main orders in which items can be auctioned. Number of buyers: in relation to the number of buyers involved, it is possible to distinguish two main types of auctions; firstly, Reverse Auctions[98] (n sellers - 1 buyer) which is a type of auction in which the roles of buyers and sellers are reversed.
Combinatorial auctions
The test bed - RoboCup Rescue
- RoboCup Rescue simulator
- Rescue agents
- Environment complexity
- Multi-agent testbed
One of the main problems with the RoboCup Rescue simulation is the partial observability of the environment. 73] presents a centralized approach applied to the RoboCup Rescue scenario [1], where the central agent (fire station) is notified of the situation, i.e.
Auctions applied to RoboCup Rescue
- Ahmed et al
- Akin and Ozkucur
- Sedaghat et al
- Adams et al
Then fire station and ambulance center collect the requests received and send a request to the police office. Police forces receive the requests and make bids on them, and send their bids to the police office.
Combinatorial auctions applied to RoboCup Rescue
Nair et al
ALADDIN is a multi-disciplinary project that develops techniques, architectures and mechanisms for multi-agent systems in uncertain and dynamic environments. Research in ALADDIN focuses on the design of auction strategies (i.e., bidding and payment strategies) and mechanisms to achieve fair, rational, and efficient system goals and coordination.
Habibi et al
Other approaches for task coordination applied to RoboCup Rescue
- Division of labor in swarms (Ferreira et al.)
- Distributed constraint optimization (Scerri et al.)
- Partially observable markov decision processes (Paquet et al.)
- Evolutionary algorithms
- Markov game formulation (Chapman et al.)
- Reinforcement learning, fuzzy and neural networks
The Swarm-GAP method was evaluated in an abstract scenario and in the RoboCup Rescue scenario. [17] reports a decentralized game-theoretic approach to distribute casualties among ambulance teams in the RoboCup Rescue simulation environment.
Challenges in market-based multi-agent/robot coordination
- Challenges in replanning
- Challenges in dinamicity
- Challenges in task allocation and sequencing
- Challenges in tight coordination
These parameters for selecting paths are for example: distances of the path to the police force, the number of buried people including civilians and agent teams buried in the buildings connected to the path and the reported blockades of the path. The study of auctions and market-based mechanisms in dynamic environments and with uncertainty is one of the challenges that Dias et al.
Final remarks
In this sense, it is necessary to create a new method of deciding which tasks the provider should compete for. In this sense, it is necessary to create and design new mechanisms for task allocation and coordination to provide agents with an efficient decision-making tool in these complex scenarios.
System requirements
For this reason, a strategy of information sharing and cooperation between heterogeneous agents is needed; this strategy must take into account the communication constraints to avoid overloading the system. The rescue agents need task allocation algorithms that take into account their preferences, for example the priority of the victims in relation to the ambulance crews' operation, and the priority or urgency of fires in the fire brigade operation.
Multi-agent task coordination
For example, ambulance teams need information from the fire brigade and police to find the victims; in the same way, the ambulance teams and fire brigades need cleared roads for the police forces. In this sense, the new algorithms should incorporate this information into the task assignment problem.
Task allocation
Example of reasoning tasks within a rescue environment
This objective refers that one agent team locating a buried or injured victim must proceed to the victim's position. Goal setting: In this part of the agent team's reasoning activities; the agent team has V under its control.
Formalization aspects
Multi-agent coordination
Where CP is the total of the capabilities of the set of agent teams. GA and GO is the total of the goals to be achieved within the cooperative scenarios. Now let's assume that a supervisor or central agent is part of a cooperative group or agent teams within the cooperative scenarios.
Agent team and task capabilities
Each goal consists of a set of tasks to be accomplished and the set of capabilities required to achieve the goal (see Figure 4.6). According to the capability matrix formulation, an AT must have some capability in its environment to accomplish a given task.
Algorithms for distributed task allocation and scheduling
- Algorithm for task allocation using direct supervision
- Algorithm for task allocation using mutual adjustment
- Sequencing tasks according due-date
- Replanning of tasks
The proposed algorithms in this chapter are motivated by the needs of task allocation and rescheduling in the concrete domain of the RoboCup Rescue. With these limitations it becomes primordial to manage the communication efficiently (See Figure 2.6) in Chapter 2.
Scheduling Algorithm for Task Allocation (SATA)
Sequencing according to due-date
Since EDD is a greedy algorithm, it is possible to simply find the first task to run without having to schedule all the tasks. This allows agents to find the next task to be performed in time O(n), specifying the number of tasks [14].
Scheduling problem when rescuing victims
This algorithm is optimal if there is no congestion, for example if it is possible to complete all the tasks in the given time. This property of the EDD algorithm is interesting in dynamic environments where the agents must respond to changes in the environment.
Victim’s death time estimation
In the RoboCup rescue scenario, a fire brigade can detect a fire within a distance roughly proportional to its fire, where D=K∗ (cycles from start of burning) The fire station agent has a better global view of situation and therefore may suggest fires to fire agents. In this sense, by using auctions, we can take advantage of the best global view of the fire station agent and the best local view of the firefighter agent.
Combinatorial Auctions for Task Allocation (CATA)
- Reverse combinatorial auction formulation
- RoboCup Rescue combinatorial auction formulation
- Single versus combinatorial auctions
- The RoboCup Rescue combinatorial auction process
- The RoboCup Rescue communication flow
- Bidding strategy
- Solving the RoboCup Rescue Winner Determination Problem (RRWDP) 93
- Other implementation issues. Dummy bids
- Rescheduling strategy
In the next section, the solution to the RoboCup Rescue winner problem is presented. Bln} be the subset of bids in the leave nodes of the left subtree of the bid tree (in branch).
Replanning implementation
In successive cycles, once R1 is cleared, the agent is informed about it by the central agent. For example, P T represents the Path's ID to be cleared and T da fire's ID that is delayed in the process explained previously.
Experimentation and results
- Death time prediction experiments
- System’s experiments with cooperation versus without cooperation
- SATA experiments
- CATA with replanning mechanism experiments
In addition, we compared these estimates with the actual time of death of the civilian. The results comparing both systems' performance (with cooperation versus without cooperation) are presented in Figure 5.35.
Final remarks
In the RoboCup Rescue scenario, the tasks are unknown at the beginning of the simulation. In addition, an important parameter that the ambulance team officers need to know is the expected time of death of the victims.
Results analysis
The main goal is to find the optimal solution and we do not care if all tasks are covered by the bids sent by the agents.
Contributions
We presented results demonstrating the efficiency of the predictions in the RoboCup Rescue simulation. A collaborative approach has been implemented that allows agents to share information and improve their state of the world.
Future work
Combinatorial auctions mechanism (CATA)
Scheduling mechanism (SATA)
Another improvement of our scheduling approach would be to consider the shift time between agent and tasks. Using better estimates for the movement times can be a first step to improve the schedules.
Replanning
In our experiments we used a constant time, but the scheduler could generate better schedules if the actual moving time was taken into account.
Publications
List of publications related to this PhD
Inverse Combinatorial Auctions for the Allocation of Resources in Rescue Scenario. In the International Conference on Planning and Scheduling (ICAPS' . 06). Silvia Su´arez, Beatriz L´opez, Josep Llu´ıs de la Rosa and Esteve del Acebo.
Other publications
Approaches to assigning and re-scheduling tasks. In Proceedings of the 24th Annual UK Planning and Designing Special Interest Group Workshop (PLANSIG'05). Cooperative strategies to strengthen civilian agents live in the RoboCup rescue simulator scenario. In Proceedings of the First International Workshop on Synthetic Simulation and Robotics for Earthquake Disaster Mitigation. About RoboCup 2003.
Configuration’s file of the Foligno’s map
Configuration’s file of the Ramdom Large’s map
Configuration’s file of the Ramdom Small’s map
- Typical structure of a multi-agent system
- RoboCup Rescue simulator architecture
- Communication between the simulator modules during the initialization phase. 22
- Viewer of the Kobe Map from the RoboCup Rescue scenario
- Communication organization. Links between different types of agents indi-
- One scheme of agent teams within the scenario S
- Environment interaction. Each agent AT i only knows a fraction S i of the
- One general scheme of a group of agent teams in a mutual adjustment process. 54
- One scheme of the tasks and the capabilities needed to perform a deter-
- Typical combinatorial auction process. It can be divided into three main
- Proposed combinatorial auction coordination process. It can be divided into
- One scheme of a cooperative group within the scenario S. Each agent team
- Replanning algorithm
- Plan for the example of victims’ rescue
- Scheduling algorithm in ambulance team agents
- Scheduling algorithm in ambulance central agent
- Sample bid configuration showing v 2 and v 3 infeasibilities
- Foligno (Italy) map of the RoboCup Rescue simulation scenario
- Kobe (Japan) map of the RoboCup Rescue simulation scenario
- Random map of the RoboCup Rescue simulation scenario
- Health points (hp) of victims on the Foligno Map for 300 cycles
- hp of victims on the Kobe Map
- hp of victims on the Random Map
- Damage to victims on all three Maps
- Trajectories with Single-Item Auctions
- Trajectories with Combinatorial Auctions
- Timeline of the task allocation process for fire brigades and fire station agents. 85
- Combinatorial auction algorithm in the rescue agent part. Communication
- Messages flow about tasks among rescue teams and central agents
- Messages flow about unblocking road tasks from police forces and fire brigades
- Auctioning of fires by fire brigades
- Bid configuration infeasibility in ambulance teams
- Bidtree for the example of victims rescue tasks
- Bidtree with increasing bid count
- Bidtree with decreasing bid count
- Algorithm to winner determination in combinatorial auctions
- First level in the search tree
- Two first solutions provided for the algorithm
- Total search space generated and solutions provided in shared square
- Search space generated for bidtree with increasing bid count
- Mechanism for replanning of tasks
- Algorithm for replanning of tasks
- Simulation results for three different methods of victims’ death time estima-
- Prediction results of the Case based (CB) mechanism. This graphic presents
- Messages flow about victims’ position from police forces and fire brigades to
- Messages flow about victims’ position among ambulance teams
- Comparison of system’s performance using cooperation versus without co-
- Comparison of system’s performance using distance versus death time cri-
- Scenarios for Kobe, Foligno, Random Large and Random Small maps
- Comparison of SATA performance in four different scenarios
- Kobe Map’s initial situation
- Comparison with RoboAkut strategy
In Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, pp. In Proceedings of the 7th International Joint Conference Autonomous Agents and Multiagent Systems: Presentation Papers, p.