Response threshold models, stochastic learning automata and ant colony optimization-based decentralized self-coordination algorithms for heterogeneous multi-tasks distribution in multi-robot systems

Texto completo

(1)UNIVERSIDAD POLITÉCNICA DE MADRID FACULTAD DE INFORMÁTICA. Response Threshold Models, Stochastic Learning Automata and Ant Colony Optimization-based Decentralized Self-Coordination Algorithms for Heterogeneous Multi-Tasks Distribution in Multi-Robot Systems. Ph.D Thesis. Alma Yadira Quiñonez Carrillo M.Sc. in Artificial Intelligence. Madrid, 2012.

(2)

(3) DEPARTAMENTO DE INTELIGENCIA ARTIFICIAL FACULTAD DE INFORMÁTICA. Response Threshold Models, Stochastic Learning Automata and Ant Colony Optimization-based Decentralized Self-Coordination Algorithms for Heterogeneous Multi-Tasks Distribution in Multi-Robot Systems. Alma Yadira Quiñonez Carrillo M.Sc. in Artificial Intelligence. Thesis Advisors Javier de Lope Asiaı́n PhD. in Informatics Darı́o Maravall Gómez-Allende PhD. Telecommunications Engineer. Madrid, 2012.

(4)

(5) Tribunal nombrado por el Magfco. y Excmo. Sr. Rector de la Universidad Politécnica de Madrid, el dı́a —– de ———– de 2012. Presidente: —————————————– Vocal:. —————————————–. Vocal:. —————————————–. Vocal:. —————————————–. Secretario: —————————————– Suplente: —————————————– Suplente: —————————————– Realizado el acto de defensa y lectura de la Tesis el dı́a —– de ———– de 2012 en la Facultad de Informática.. VOCAL. VOCAL. PRESIDENTE. VOCAL. SECRETARIO. v.

(6)

(7) I would like to dedicate this thesis to my Mother and my Brothers..

(8) Acknowledgements After such a great experience, I obviously have many people to thank... First, I want to thank all my family members. Thanks for being there and supporting me in every decision. Thank you for believing in me and giving me the strength to face even the most difficult things. Definitely thanks to you all I was able achieve this objective. I would also like to take this opportunity to thank my supervisors, Javier de Lope y Darı́o Maravall, because they have helped me enormously to further my understanding and expand my horizons in the field of robotics, but above all, I am very grateful to them for their unfailing interest, guidance and wisdom during the development this project. I am sincerely thankful with the Consejo Nacional de Ciencia y Tecnologı́a, the Univesidad Autónoma de Sinaloa and the Universidad Politécnica de Madrid for contributing with the financial support in conducting this PhD thesis. A heartfelt thanks also to the members of the Lab for making my stay more comfortable, but in particular, to Antonio Fernández and Juan Bekios for their comments and suggestions. Finally, but not least important, I would like to express my gratitude to all my friends that I met here in Madrid, who they not only encouraged me during the research career, but also, have given me many great moments. Thanks to Marinela, Iván, Lindsay, Miguel, Jez, Boris, Gonzalo, Juan, Tony, Ernesto, Raúl, Ghislain, David and Monse for sharing with me so many lunches and speaking not only about work, I have enjoyed these last few years enormously! Thank you all for your support, friendship and conviviality. Yadira Quiñonez. viii.

(9) Abstract In recent decades, there has been an increasing interest in systems comprised of several autonomous mobile robots, and as a result, there has been a substantial amount of development in the field of Artificial Intelligence, especially in Robotics. There are several studies in the literature by some researchers from the scientific community that focus on the creation of intelligent machines and devices capable to imitate the functions and movements of living beings. Multi-Robot Systems (MRS) can often deal with tasks that are difficult, if not impossible, to be accomplished by a single robot. In the context of MRS, one of the main challenges is the need to control, coordinate and synchronize the operation of multiple robots to perform a specific task. This requires the development of new strategies and methods which allow us to obtain the desired system behavior in a formal and concise way. This PhD thesis aims to study the coordination of multi-robot systems, in particular, addresses the problem of the distribution of heterogeneous multi-tasks. The main interest in these systems is to understand how from simple rules inspired by the division of labor in social insects, a group of robots can perform tasks in an organized and coordinated way. We are mainly interested on truly distributed or decentralized solutions in which the robots themselves, autonomously and in an individual manner, select a particular task so that all tasks are optimally distributed. In general, to perform the multi-tasks distribution among a team of robots, they have to synchronize their actions and exchange information. Under this approach we can speak of multi-tasks selection instead of multi-tasks assignment, which means, that the agents or robots select the tasks instead of being assigned a task by a central controller. The key element in these algorithms is the estimation. ix.

(10) of the stimuli and the adaptive update of the thresholds. This means that each robot performs this estimate locally depending on the load or the number of pending tasks to be performed. In addition, it is very interesting the evaluation of the results in function in each approach, comparing the results obtained by the introducing noise in the number of pending loads, with the purpose of simulate the robot’s error in estimating the real number of pending tasks. The main contribution of this thesis can be found in the approach based on self-organization and division of labor in social insects. An experimental scenario for the coordination problem among multiple robots, the robustness of the approaches and the generation of dynamic tasks have been presented and discussed. The particular issues studied are: • Threshold models: It presents the experiments conducted to test the response threshold model with the objective to analyze the system performance index, for the problem of the distribution of heterogeneous multitasks in multi-robot systems; also has been introduced additive noise in the number of pending loads and has been generated dynamic tasks over time. • Learning automata methods: It describes the experiments to test the learning automata-based probabilistic algorithms. The approach was tested to evaluate the system performance index with additive noise and with dynamic tasks generation for the same problem of the distribution of heterogeneous multi-tasks in multi-robot systems. • Ant colony optimization: The goal of the experiments presented is to test the ant colony optimization-based deterministic algorithms, to achieve the distribution of heterogeneous multi-tasks in multi-robot systems. In the experiments performed, the system performance index is evaluated by introducing additive noise and dynamic tasks generation over time.. x.

(11) Resumen En las últimas décadas, ha habido un interés creciente en los sistemas compuestos por varios robots móviles autónomos, y como resultado, ha surgido una cantidad sustancial de desarrollo en el campo de la inteligencia artificial, especialmente en la robótica. Hay varios estudios en la literatura por parte de algunos investigadores de la comunidad cientı́fica que se centran en la creación de máquinas inteligentes y dispositivos capaces de imitar las funciones y los movimientos de los seres vivos. En los sistemas multi-robot (MRS) a menudo pueden tratar con tareas que son difı́ciles, por no decir imposibles, de realizar por un solo robot. En el contexto de los MRS, uno de los principales retos es la necesidad de controlar, coordinar y sincronizar el funcionamiento de múltiples robots para realizar una tarea especı́fica. Esto requiere el desarrollo de nuevas estrategias y métodos que permitan obtener el comportamiento deseado del sistema de una manera formal y concisa. Esta tesis tiene como objetivo el estudio de la coordinación de sistemas multirobot, en particular, aborda el problema de la distribución de múltiples tareas heterogéneas. El principal interés por este tipo de sistemas es comprender cómo a partir de reglas sencillas inspiradas en la división del trabajo en los insectos sociales, un grupo de robots pueden realizar tareas de una manera organizada y coordinada. Estamos interesados principalmente en soluciones verdaderamente distribuidas o descentralizadas en el que los propios robots, de forma autónoma y de manera individual, seleccionan una tarea particular de tal modo que todas las tareas se distribuyan de manera óptima. En general, para realizar la distribución de múltiples tareas entre un equipo de robots, tienen que sincronizar sus acciones e intercambiar información. Bajo este enfoque se puede hablar de la selección de múltiples tareas en lugar de la. xi.

(12) asignación de múltiples tareas, es decir, cómo los agentes o robots seleccionan las tareas en lugar de ser asignados a una tarea por un controlador central. El elemento fundamental en estos algoritmos es la estimación de los estı́mulos y la actualización adaptativa de los umbrales. Esto significa que cada robot realiza dicha estimación de forma local dependiendo de la carga o el número de tareas pendientes por ejecutar. Además, es muy interesante la evaluación de los resultados en función de cada enfoque comparando los resultados obtenidos mediante la introducción de ruido en el número de cargas pendientes para simular el error del robot en la estimación del número real de tareas pendientes. La principal aportación de esta tesis se puede encontrar en un enfoque basado en la auto-organización y división del trabajo en los insectos sociales. Un escenario experimental para el problema de la coordinación entre múltiples robots, la robustez de los enfoques y la generación de tareas dinámicas han sido presentados y discutidos. Los temas especı́ficos estudiados son los siguientes: • Modelos de umbral: se presentan los experimentos realizados para probar el modelo umbral de respuesta con el objetivo de analizar el ı́ndice de rendimiento del sistema, para el problema de la distribución de múltiples tareas heterogéneas en los sistemas multi-robot; también se ha introducido ruido aditivo en el número de cargas pendientes y se han generado tareas dinámicas a través del tiempo. • Métodos de autómatas de aprendizaje: se describen los experimentos para probar los autómatas de aprendizaje basadas en algoritmos probabilı́sticos. El enfoque fue probado para evaluar el ı́ndice de rendimiento del sistema con ruido aditivo y la generación de tareas dinámicas para el mismo problema de la distribución de múltiples tareas heterogéneas en los sistemas multi-robot. • Optimización de colonias de hormigas: el objetivo de los experimentos presentados es poner a prueba el algoritmo de optimización de colonias de hormigas basado en algoritmos deterministas, para lograr la distribución de múltiples tareas heterogéneas en los sistemas multi-robot. En los experimentos realizados se evaluó el ı́ndice de rendimiento del sistema mediante la introducción de ruido aditivo y la generación de tareas dinámicas en el tiempo.. xii.

(13) Contents Acknowledgements. viii. Abstract. ix. Resumen. xi. Contents. xiii. List of Figures. xviii. List of Tables. I. xx. Goals and Background. 1. 1 Introduction 1.1 Motivation . . . . . . . . . . . . . . . 1.2 Thesis Objectives . . . . . . . . . . . 1.2.1 General Objective . . . . . . . 1.2.2 Specific Objectives . . . . . . 1.3 Main Contributions and Publications 1.3.1 Main Contributions . . . . . . 1.3.2 Publications . . . . . . . . . . 1.4 Thesis Structure . . . . . . . . . . . .. . . . . . . . .. 2 3 4 5 5 6 6 7 8. 2 State of the Art 2.1 Multi-Robot Systems . . . . . . . . . . . . . . . . . . . . . . . . .. 11 12. xiii. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . . . . . . ..

(14) CONTENTS. 2.1.1 2.1.2. 2.2. 2.3. II. Coordination in Multi-Robot Systems . . . . . . . . . Architectures for Multi-robot Systems . . . . . . . . 2.1.2.1 Centralized Architectures . . . . . . . . . . 2.1.2.2 Hierarchical Architectures . . . . . . . . . . 2.1.2.3 Decentralized Architectures . . . . . . . . . 2.1.2.4 Hybrid Arquitectures . . . . . . . . . . . . . 2.1.3 Main Problems among a Group of Robots . . . . . . 2.1.4 Coordination Schemes: Cooperative and Competitive Fields of Application . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Cooperative Manipulation . . . . . . . . . . . . . . . 2.2.2 Unstructured Environments . . . . . . . . . . . . . . 2.2.3 Formation Control . . . . . . . . . . . . . . . . . . . 2.2.4 Biologically-Inspired . . . . . . . . . . . . . . . . . . Previous and Related Work . . . . . . . . . . . . . . . . . . 2.3.1 Formal Methods in Relation to Coordination . . . . . 2.3.1.1 Multi-Agent Systems . . . . . . . . . . . . . 2.3.1.2 Swarm Robots . . . . . . . . . . . . . . . . 2.3.1.3 Multi-Robot Systems . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . .. Setting the Problem. 14 16 16 16 17 17 19 20 21 22 23 24 25 26 26 27 27 28. 32. 3 Problem Description 3.1 Problem Statement . . . . . . . . . . 3.2 Formal description of the problem . . 3.3 Application Scenario . . . . . . . . . 3.4 Description of the Proposed Solution. III. . . . . . . . . . . . . . . . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. Foundations. 33 34 34 35 35. 40. 4 Theoretical Fundamentals 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Threshold Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 An Overview of Response Threshold Model . . . . . . . .. xiv. 41 42 44 44.

(15) CONTENTS. 4.3. 4.4. IV. 4.2.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Learning Automata Methods . . . . . . . . . . . . . . . . . . . . . 4.3.1 A Brief Introduction . . . . . . . . . . . . . . . . . . . . . 4.3.2 Definition of Stochastic Processes . . . . . . . . . . . . . . 4.3.3 Basic Definition of Learning Automata . . . . . . . . . . . 4.3.4 Stochastic Reinforcement Algorithms based on Reward and Penalty . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ant Colony Optimization . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 A Brief Introduction . . . . . . . . . . . . . . . . . . . . . 4.4.2 Biological Inspiration . . . . . . . . . . . . . . . . . . . . . 4.4.3 The Ant System Approach . . . . . . . . . . . . . . . . . .. Experimentation and Conclusions. 5 Experimental Results 5.1 Preliminaries of the Experimentation . . . . . . . . . . . . . . . . 5.1.1 Evaluation of the Performance Index . . . . . . . . . . . . 5.1.1.1 Additive Noise Generation . . . . . . . . . . . . . 5.1.1.2 Dynamic Tasks Generation . . . . . . . . . . . . 5.2 Experiments with Threshold Models . . . . . . . . . . . . . . . . 5.2.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 Evaluation of the Approach with Additive Noise . . . . . . 5.2.3 Evaluation of the Approach with dynamic tasks . . . . . . 5.2.4 Results and Discussion . . . . . . . . . . . . . . . . . . . . 5.3 Experiments with Learning Automata-based Probabilistic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.2 Evaluation of the Approach with Additive Noise . . . . . . 5.3.3 Evaluation of the Approach with Dynamic Tasks . . . . . 5.3.4 Results and Discussion . . . . . . . . . . . . . . . . . . . . 5.4 Experiments with Ant Colony Optimization-based Deterministic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. xv. 45 48 48 49 51 52 54 54 56 58. 61 62 63 63 64 64 65 65 66 67 67 68 68 68 69 71 71 71.

(16) CONTENTS. 5.4.2 5.4.3 5.4.4. Evaluation of the Approach with Additive Noise . . . . . . Evaluation of the Approach with Dynamic Tasks . . . . . Results and Discussion . . . . . . . . . . . . . . . . . . . .. 71 73 74. 6 Conclusions and Further Work 6.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Future Research Work . . . . . . . . . . . . . . . . . . . . . . . .. 77 78 80. Bibliography. 83. xvi.

(17)

(18) List of Figures 2.1 2.2 2.3. 2.6. Taxonomy: coordination dimensions in multi-robot systems . . . . Multi-robot system . . . . . . . . . . . . . . . . . . . . . . . . . . Box-Pushing Mission [59; 107; 160; 166] and group of mobile robots designed to work cooperatively lifting columns (http://birg.epfl. ch/page28710.html) . . . . . . . . . . . . . . . . . . . . . . . . . Exploration in unstructured environments. (a) The Mars exploration rovers, Spirit and Opportunity, with a manipulator arm in front, (b) a conceptual drawing for robotic rescue of Hubble space telescope, (c) The Pathfinder rover, Sojourner and (d) Rocky 4. . Formation Control. (a) Flying in Formation Takes Aircraft Farther, Dylan Ashe (http://www.popsci.com/). In (b) shows image of Vicon cameras overlooking a group of Khepera III robots. 3 cameras shown, 8 cameras total [98] . . . . . . . . . . . . . . . . Bio-inspired robotics . . . . . . . . . . . . . . . . . . . . . . . . .. 25 26. 3.1 3.2. Experimental scenario . . . . . . . . . . . . . . . . . . . . . . . . Procedure for the selection of multi-tasks . . . . . . . . . . . . . .. 36 38. 4.1 4.2. Threshold function . . . . . . . . . . . . . . . . . . . . . . . . . . Semi-logarithmic plot with different thresholds (θ = 1, 5, 20, 50) and with n = 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interaction of learning automaton with random environment . . . In [12] presents a experimental setting that shows the shortest path finding capability of ant colonies . . . . . . . . . . . . . . . . . . .. 45. 2.4. 2.5. 4.3 4.4. xviii. 14 21. 23. 24. 46 52 55.

(19) LIST OF FIGURES. 5.1. Learning curves with the evolution of the system performance index for self-election of tasks using Response Threshold Models with noise = 0.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Learning curves with the evolution of the system performance index for self-election of tasks using Response Threshold Models with noise = 0.25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Dynamic tasks generation: learning curves with the evolution of the system performance index for self-election of tasks using Response Threshold Models . . . . . . . . . . . . . . . . . . . . . . . 5.4 Learning curves with the evolution of the system performance index for self-election of tasks using Learning Automata-based probabilistic algorithms with noise = 0.10 . . . . . . . . . . . . . . . . 5.5 Learning curves with the evolution of the system performance index for self-election of tasks using Learning Automata-based probabilistic algorithms with noise = 0.25 . . . . . . . . . . . . . . . . 5.6 Dynamic tasks generation: learning curves with the evolution of the system performance index for self-election of tasks using Learning Automata-based probabilistic algorithms . . . . . . . . . . . . 5.7 Learning curves with the evolution of the system performance index for selfelection of tasks using Ant Colony Optimization-based deterministic algorithms with noise = 0.10 . . . . . . . . . . . . . 5.8 Learning curves with the evolution of the system performance index for selfelection of tasks using Ant Colony Optimization-based deterministic algorithms with noise = 0.25 . . . . . . . . . . . . . 5.9 Dynamic tasks generation: learning curves with the evolution of the system performance index using Ant Colony Optimizationbased deterministic algorithms . . . . . . . . . . . . . . . . . . . . 5.10 The index k represents the number of tasks expected to be generated during a time interval for different values of λ and P (X = k) describes the probability that a value of variable X with a given probability distribution is equal to k . . . . . . . . . . . . . . . . 5.11 Number of tasks performed by each robots . . . . . . . . . . . . .. xix. 66. 67. 68. 69. 70. 70. 72. 72. 73. 74 75.

(20) List of Tables 2.1. Taxonomies multi-robot . . . . . . . . . . . . . . . . . . . . . . .. 5.1. Experiments performed without dynamic tasks and their respective variants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Experiments performed with dynamic tasks and their respective variants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 5.2. xx. 15. 64 65.

(21)

(22) Part I Goals and Background. 1.

(23) Chapter 1 Introduction A rational and fruitful discussion is impossible unless the participants share a common framework of basic assumptions or, at least, unless they have agreed on such a framework for the purpose of the discussion. Karl R. Popper. SUMMARY: This chapter details the aspects related to the research area. Section 1.1 mentions the reasons that justify why it is important to develop this research work. Section 1.2 defines the general and specific objectives. Section 1.3 presents the main contribution of the thesis and presents the results obtained that have been presented in several international conferences and published in international scientific journals with peer reviewed. Finally, section 1.4 briefly describes the organization of the thesis.. 2.

(24) 1.1. MOTIVATION. 1.1. Motivation. The systems formed by multiple mobile robots, also known as Multi-Robot Systems (MRS) are employed for different reasons, however, one of the main motivations is that MRS can be used to increase the system effectiveness in terms of time and quality, providing greater flexibility in the tasks execution. Generally speaking, the term multi-robot system includes different types of robotic systems, for example, several industrial manipulators, mobile robots with manipulators on board, or team of autonomous vehicles, but, in this thesis, the term will be used to refer to a team of cooperating mobile robots to carry out the distribution of heterogeneous multi-tasks. The problem of coordination in MRS has been discussed in the literature in many forms; each of the proposed methods are applied for groups of robots that work closely together to accomplish a task composed of multiple sub-tasks. As is typical for many complex systems, mathematical models are needed to obtain tradeoff and accuracy in a system. The main benefits or advantages of these systems are that the robots are capable of performing multiple tasks with much greater precision than humans, but mostly because they can be extremely efficient, they can perform calculations quickly, they can minimize risk and also complete a task in less time. Probably one of the most promising directions for research in this area is based on the coordination of multiple robots. In recent decades, there has been a large amount of research done with respect to autonomous mobile robots related to the coordination between them [3; 18; 24; 75]. These investigations have been directed toward finding efficient and robust methods for controlling these groups of mobile robots. With this increase there has also arisen new problems that require the execution of bigger and more complex tasks. A very useful solution to this problem is to implement multiple cooperative robots to accomplish a certain task since the cost is generally lower for several robots than it would be for one single robot. In addition, a group of robots is capable to perform many tasks as well as faster than a single independent robot could ever do. For example, a group of unmanned aerial vehicles (UAVs) can be deployed to perform dangerous tasks to improve the chance of success and to study the conse-. 3.

(25) 1.2. THESIS OBJECTIVES. quences in case of a natural disaster. In some applications such as reconnaissance missions, mine detection, surveillance and rescue victims, groups of robots can augment and even replace humans in order to avoid possible injury to those that protect us. During these missions, it is necessary to maintain communication within the team of robots to carry out successfully the task at hand. MRS can often deal with tasks that are difficult, if not impossible, to be accomplished by a single robot. In the context of MRS, one of the challenges is the need to control, coordinate and synchronize the operation of multiple robots to perform a specific task. This requires the development of new strategies and methods to obtain the desired system behavior, by means of, simple rules inspired by the division of labor in social insects, in order that a group of robots can perform tasks in an organized and coordinated way.. 1.2. Thesis Objectives. This PhD thesis focuses on the self-coordination problem of MRS and in particular addresses the distribution of heterogeneous multi-tasks in a robust and efficient manner. We take into account a specifically distributed or decentralized approach as we are particularly interested in experimenting with truly autonomous and decentralized techniques in which the robots themselves are responsible for choosing a particular task in an autonomous and individual way. In this regard, we have experimented with different techniques: firstly, the application of the response threshold models inspired by division of labor in social insects, secondly, the application of the reinforcement learning algorithm based on learning automata theory, and finally, ant colony optimization-based deterministic algorithms. There are different strategies to address the task assignment problem, but in this thesis is presented different approaches based on self-organizing and biologically inspired to address the multi-tasks selection instead of multi-tasks assignment. This thesis will attempt, first, to answer the following questions: • It is posible that agents or robots select the tasks instead of being assigned? • It is posible to obtain an optimal distribution of the tasks by introducing noise in the approaches?. 4.

(26) 1.2. THESIS OBJECTIVES. 1.2.1. General Objective. The main goal of this PhD thesis is: “Study, analyze and propose a set of techniques or methods for the problem of coordinating multi-robot systems, specifically in the distribution of heterogeneous multi-tasks, and experimenting with different approaches based chiefly on self-organization and emergence that is biologically inspired.”. 1.2.2. Specific Objectives. The main goal is decomposed into several objectives, then, we establish the following specific objectives for this research: • Investigate decentralized approaches inspired by the division of labor in social insects and apply to the problem of distribution of heterogeneous multi-tasks in MRS. • Define the experimental scenario. • Define the number of robots and the number of tasks in the system. • Design the auto-assignment algorithm for multi-tasks with response threshold models. • Design the auto-assignment algorithm for multi-tasks by the reinforcement learning algorithm based on learning automata theory. • Design the auto-assignment algorithm for multi-tasks using ant colony optimizationbased deterministic algorithms. • Analyze the robustness of the approaches by introducing noise to the methods. • Generate dynamic tasks over time.. 5.

(27) 1.3. MAIN CONTRIBUTIONS AND PUBLICATIONS. 1.3 1.3.1. Main Contributions and Publications Main Contributions. The thesis presents several contributions to the self-coordination problem of multi-robot systems in the distribution of heterogeneous multi-tasks with different approaches biologically-inspired. Therefore, the results obtained are based on papers written that have been presented and published in several international conferences and journals. The main contributions of the thesis are: • A bio-inspired solution based on response threshold models to solve the problem for self-coordination of multi-robots, through the distribution of heterogeneous and specialized multi-tasks in multi-robot systems. • A solution through automata learning-based probabilistic algorithm, that focuses on the general problem of coordinating multiple robots, specifically, for the self-coordination in the selection of heterogeneous multi-tasks in multi-robot systems. • A solution using two different approaches by applying ant colony optimizationbased deterministic algorithms as well as learning automata-based probabilistic algorithms which addresses the general problem of coordinating multiple robots specifically for decentralized distribution of multi-tasks in heterogeneous robot teams. • A solution using two different approaches by applying response threshold models and stochastic learning automata to solve the problem corresponding to self-coordination in the distribution of heterogeneous multi-tasks in multi-robot systems. • An experimental scenario for all approaches has been proposed in order to analyze the coordination problem among multiple robots. The robustness of each method has been studied by the introduction of noise, which perturbs. 6.

(28) 1.3. MAIN CONTRIBUTIONS AND PUBLICATIONS. the number of pending load. The performance index with generation of tasks over time has also been analyzed.. 1.3.2. Publications. The results presented have influenced the contents of this thesis and have been published in several international conferences and journals. The research results have been published in the IEEE library, the ACM library, the ISI Web of Knowledge, Lecture Notes in Computer Science and Lecture Notes in Artificial Intelligence by Springer-Verlag. The publications are documented in the following works: Journals Publications: • De Lope, J., Maravall, D. and Quiñonez, Y. (2012). Response threshold models and stochastic learning automata for self-coordination of heterogeneous multi-tasks distribution in multi-robot systems. Robotics and Autonomous Systems - Impact Factor: 1.313 [31]. International Conferences Publications: • Quiñonez, Y., De Lope, J. and Maravall, D. (2009). Communication and coordination of robots teams in dynamic environments. Twelve International Conference on Computer Aided Systems Theory, EUROCAST 2009, pp. 150–151 [128]. • Quiñonez, Y., Baca, J., De Lope, J., Ferre, M. and Aracil, R. (2010). Selfalignment approach based on cooperative behaviors for the docking process of modular mobile robots. IEEE International Conference on Electronics, Robotics and Automotive Mechanics, CERMA 2010, pp. 445–450 [130]. • Quiñonez, Y., Maravall, D. and De Lope, J. (2012). Application of selforganizing techniques for the distribution of heterogeneous multi-tasks in multi-robot systems. IEEE International Conference on Electronics, Robotics and Automotive Mechanics, CERMA 2012, pp. 66–71 [133].. 7.

(29) 1.4. THESIS STRUCTURE. Book Chapters Publications • Quiñonez, Y., De Lope, J. and Maravall, D. (2009). Cooperative and competitive behaviors in a multi-robot system for surveillance tasks. Computer Aided Systems Theory, EUROCAST 2009. Revised Selected Papers, LNCS 5717. R. Moreno-Diaz, F. Pichler, A. Quesada (Eds.) Springer-Verlag, Berlin Heidelberg, pp. 437–444 [129]. • Quiñonez, Y., De Lope, J. and Maravall, D. (2011). Bio-inspired decentralized self-coordination algorithms for multi-heterogeneous specialized tasks distribution in multi-robot systems. Foundations on Natural and Artificial Computation, LNCS 6686. J.M. Ferrández et al. (Eds.) Springer-Verlag, Berlin Heidelberg, pp. 30–39 [131]. • Quiñonez, Y., De Lope, J. and Maravall, D. (2011). Stochastic learning automata for self-coordination in heterogeneous multi-tasks selection in multi-robot systems. International Conference on Advances in Artificial Intelligence, MICAI 2011, Part I, LNAI 7094, pp. 443–453 [132]. • De Lope, J., Maravall, D. and Quiñonez, Y. (2012). Decentralized multitasks distribution in heterogeneous robot teams by means of ant colony optimization and learning automata. International Conference on Hybrid Artificial Intelligence Systems, HAIS 2012, Part I, LNCS 7208, pp. 103–114 [32].. 1.4. Thesis Structure. This document is organized by a set of chapters whose contents are described briefly as follows. • Chapter 2. State of the Art This chapter explains the main features that present the systems formed by multiple robots, also it introduces an overview of previous related work on this research, in order to cover all the necessary knowledge and contextualize the associated domains. It presents an overview on the main issues of. 8.

(30) 1.4. THESIS STRUCTURE. multi-robot systems, control architectures, coordination schemes and main problems between theses systems. In addition, it provides some applications of robotics that involve different fields using multiple robots, such as: cooperative manipulation, unstructured environments, formation control and biologically-inspired. Finally, it describes briefly the main previous works related with the multi-robot systems and formal methods used. • Chapter 3. Problem Description This chapter defines the problem statement of the thesis, it presents a formal description of the problem and describes the experimental scenario. Finally, it details the description of the proposed solution to the previously defined problems. • Chapter 4. Theoretical Fundamentals This chapter some mathematical concepts used throughout the thesis are reviewed. The main objective of the chapter is to describe mathematical models or probabilistic based on distributed or decentralized approaches inspired by division of labor in social insects. It presents a brief introduction about mathematical models. Firstly, it describes an overview of response threshold model and specifically a description of mathematical model of response thresholds. Secondly, it presents a brief introduction about learning automata methods, basic definitions of the theory of stochastic processes, a basic definition of learning automata and stochastic reinforcement algorithms based reward and penalty. And finally, it describes a brief introduction of the ant colony optimization, biological inspiration and description of the ant system algorithm. • Chapter 5. Experimental Results This chapter we present the experimental results obtained from the applying of the different decentralized approaches inspired on division of labor in social insects, such as: the response threshold model, ant colony optimizationbased deterministic algorithms and the learning automata-based probabilistic algorithms. We analyze the results of experimentation, evaluating the performance index by introducing additive noise to the number of pending. 9.

(31) 1.4. THESIS STRUCTURE. loads and we generated dynamic tasks over time. • Chapter 6. Conclusions and Further Work This chapter we present the conclusions of the thesis, and finally, are detailed the future research lines derived from this research work.. 10.

(32) Chapter 2 State of the Art Science, despite its incredible advances, is not and will never be able to explain everything. It will continue to conquer new areas that today are beyond our understanding. But the frontiers of knowledge, however high these may be raised, will always have an infinite world of mystery. Gregorio Maran. SUMMARY: This chapter explains the main features that present the systems formed by multiple robots, also it introduces an overview of previous related work on this research, in order to cover all the necessary knowledge and contextualize the associated domains. Section 2.1 provides an overview on the main issues of multi-robot systems, control architectures, coordination schemes and main problems between theses systems. Section 2.2 presents some applications of robotics that involve different fields using multiple robots, such as: cooperative manipulation, unstructured environments, formation control and biologically-inspired. Finally, section 2.3 serves as an presentation and review of the main previous works related with the multi-robot systems and formal methods used.. 11.

(33) 2.1. MULTI-ROBOT SYSTEMS. 2.1. Multi-Robot Systems. MRS is one of the characteristic applied areas of Artificial Intelligence that has gotten an amazing growth since its inception until today [50; 55], and it has developed very significant progress in various fields of application [124], becoming a fundamental tool to produce, work and perform dangerous jobs on earth and beyond. In recent years, MRS are increasingly used in highly dynamic or contradictory environment to deal with complex tasks [83], are quickly becoming a vast research area and includes several different topics and ideas, as shown in the various works [4; 35; 49; 65; 80; 122]. A MRS consists of a set of robots that, in the same environment, interact with each other to achieve a common goal [53], thus trying to improve the effectiveness, performance and robustness. These systems provide greater flexibility in performing tasks and possible fault tolerance. To achieve that several robots coordinate with each other to perform a specific mission is not a trivial task, because, they must be designed to operate in dynamic environments in which we must also take into account the classical problems of autonomous robotics (e.g. uncertainty and unforeseen changes always present), new difficulties arising from the influence of the team robots on the environment and the task goal. The main advantages of these systems with regard to a single robot is that they have higher flexibility, efficiency and reliability achieving a more robust behavior by accomplishing coordinated tasks that are not possible for single robots; they can perform complex tasks much faster and execute tasks beyond the limits of single robots. In fact, a multi-robot system may result robust to malfunctions like unreliable communication and robot failures. Arai et al. [4] and Parker [122] have identified the following primary research topics within MRS: • biological inspirations; • communication; • architectures, task allocation, and control; • localization, mapping, and exploration;. 12.

(34) 2.1. MULTI-ROBOT SYSTEMS. • object transport and manipulation; • motion coordination; • reconfigurable robots; • learning During these years, the scientific community has developed some research progress in cooperative robotics with respect to mechanisms for coordination and communication [85]. Dudek et al. [49] present a taxonomy for multi-agent robotic systems, where proposed a classification based on the size of the team, communication parameters (communication range, bandwidth and topology), the reconfigurability of the team, the processing capacity of each member and the team composition (homogeneous vs. heterogeneous robots). A taxonomy for the classification of coordination approaches in MRS have proposed in [53; 80]. They present a classification based on different levels of coordination (unaware, aware but non coordinated, weakly coordinated, strongly coordinated systems) and is characterized by two groups of dimensions, that is the coordination dimension (cooperation, knowledge, coordination and organization) and the system dimension (communication, team composition, system architecture and team size). The term dimension refers to specific features that are grouped together in the taxonomy. Fig. 2.1 shows a hierarchical structure for the coordination dimensions of the taxonomy. The different levels of the structure are: A cooperation level, a knowledge level, a coordination level, and an organization level. The first level of the taxonomy is concerned with the ability of the system to cooperate in order to accomplish a specific task. The second level is concerned with how much knowledge each robot in the system has about the presence of other robots. The third level is concerned with the mechanism that is used in order to achieve cooperation in the system. The fourth level is concerned with the way the decision system is realized within the MRS. Finally, the work in [66] have presented a taxonomy based on coordination mechanisms and on multi-robot task allocation.. 13.

(35) Strongly Centralized. Strongly Coordinated. Weakly Coordinated. Not Coordinated. Weakly Centralized. Distributed. Knowledge. Unware. Coorination. Aware. Organization. Cooperative. Cooperation. 2.1. MULTI-ROBOT SYSTEMS. Figure 2.1: Taxonomy: coordination dimensions in multi-robot systems Some researchers have proposed taxonomies or classification systems that allow to organize and to control a multi-robot system. Then, in table 2.1 describes a summary with the most significant features of some taxonomies multi-robot presented in the literature.. 2.1.1. Coordination in Multi-Robot Systems. Coordination is the act of organizing a group of mobile robots that is of fundamental importance for any MRS. That is, coordination in MRS imply that a group of robots working together to accomplish specified actions simultaneously that can result in the completion of an overall system goal at the global-level. Cooperation refers to the simultaneous action of two or more agents that work together and produce the identical effect. In the context of multi-robot systems cooperation is defined as constructive and synergistic interaction of robots in a system to exchange information in an intelligent manner and thus achieve the execution of tasks more quickly and efficiently. In [80; 87] present a explicit defi-. 14.

(36) 2.1. MULTI-ROBOT SYSTEMS. Taxonomy Yuta et al. [169]. Domain Multi-robot. Fulbright et al. [60]. Cao [19] Balch [8]. Stone et al. [145]. Todt [154]. Description Defined from the objectives and mechanisms of decision. Multi-agent Establishes three classifications according the coupling of agents. Cooperative robots Based on problems and solutions of the cooperation. Multi-robot Useful in systems that employ reinforcement learning (tasks and rewards). Multi-agent Study the homogeneity of the agents and their level of communication. Multi-robot Based on coordination between robots.. Table 2.1: Taxonomies multi-robot. nition about cooperation and coordination in a MRS as follows: COORDINATION: Cooperation in which the actions performed by each robot. take into account the actions executed by the other robots in such a way that the whole ends up being a coherent and high performance operation. COOPERATION: Situation in which several robots operate together to per-. form some global task that either cannot be achieved by a single robot, or whose execution can be improved by using more than one robot, thus obtaining higher performances. Coordination is an essential characteristic between a groups of robot and is an important issue of investigation [84], because, they require the development of new techniques for control and coordination that enable the interaction between them and with environment to solve problems together. The coordination between the robots can vary but there are usually four kinds of architectures for coordinating of multi-robots, which are centralized, distributed, hierarchical and. 15.

(37) 2.1. MULTI-ROBOT SYSTEMS. Hybrid architectures.. 2.1.2. Architectures for Multi-robot Systems. Robot architectures are designed to facilitate the concurrent execution of taskachieving behaviors. At a very low level, robots must be able to react quickly to dynamic changes in the environment and perform reactive routines in order to accomplish tasks such as obstacle avoidance. At higher level, robots must be able to coordinate with each other, performing asynchronous tasks such as cooperative search or highly synchronized tasks such as cooperative transportation. Several different kinds of control architectures for MRS have been presented in literature, however, the main distinction can be done between centralized, hierarchical, decentralized, and hybrid [124]. 2.1.2.1. Centralized Architectures. Centralized multi-robot systems were developed as a method to coordinate communication between robots and the system. Centralization allows the main processing and computational requirements to be removed from the individual robots, and be completed on an external computer [149]. In centralized systems, a central unit collects and manages information about the environment and optimize the coordination among the robots to ensure the proper achievement of the mission; moreover, they can easily manage faults of some of the robots. In these approaches, the central unit plays a key role, because it handles the whole system, that is, it has to coordinate the information received by the sensors and manage global information of the environment, to take all possible decisions and to communicate with all robots of team, therefore, must be powerful enough to satisfy all technological requirements. 2.1.2.2. Hierarchical Architectures. Hierarchical architectures are realistic for some applications. In this control approach, each robot oversees the actions of a relatively small group of other robots, each of which in turn oversees yet another group of robots, and so forth, down to the lowest robot, which simply executes its part of the task. This architecture. 16.

(38) 2.1. MULTI-ROBOT SYSTEMS. scales much better than centralized approaches, and is reminiscent of military command and control. A point of weakness for the hierarchical control architecture is recovering from failures of robots high in the control tree [124]. 2.1.2.3. Decentralized Architectures. In Decentralized control architectures, the act of coordination is significantly more complex [170]. Decentralized multi robot systems have stemmed from the inability to adapt a fully centralized system to specific environments. Often the ability to develop a fully centralized system is difficult due to the number of robots or the capabilities of the central processor [99] and therefore decentralized systems are needed. These systems are highly scalable to large multi-robot systems and applicable to outdoor unknown environments [25]. Decentralized systems can easily result tolerant to possible faults, however, one major drawback of decentralized systems is the complexity of the communications network that needs to be developed between the robots [90], since each robot works independently because the resources are distributed among all the robots. Each robot uses its own sensors to extrapolate local information of the environment and the relative position of the robots closest to take its own decisions; that is, it is more difficult to coordinate the robots and optimize the execution of the mission, then, a lot of cooperation should be developed for that the system can work together. 2.1.2.4. Hybrid Arquitectures. Hybrid control architectures combine local control with higher-level control approaches to achieve both robustness and the ability to influence the entire team’s actions through global goals, plans, or control. Many multi-robot control approaches make use of hybrid architectures [124]. For these schemes have been proposed several works in the literature with experiments on coordination of multi-robot systems [24; 75; 88; 105]. There are several examples of different multi-robot specific architectures, employing different control strategies. Below we brief describe three prominent architectures that have been proposed in literature: 1. The ALLIANCE architecture has been developed by Parker [121], is a. 17.

(39) 2.1. MULTI-ROBOT SYSTEMS. control architecture for fault tolerant, reliable and adaptive to cooperative control of teams of heterogeneous mobile robots performing missions composed of loosely coupled subtasks that may have ordering dependencies. ALLIANCE is a fully distributed, behavior-based architecture that incorporates the use of mathematically-modeled motivations. The ALLIANCE architecture is implemented on each robot in the cooperative team, delineates several behavior sets, each of which correspond to some high-level task-achieving function. The primary mechanism enabling a robot to select a high-level function to activate is the motivational behavior. 2. The Layered Architecture for coordination of mobile robots was developed by Simmons et al. [144], is an architecture that enables multiple robots to explicitly coordinate actions at multiple levels of abstraction. Their layered architecture has three layers than enables robots to interact directly at the behavioral level, the executive level and the planning level. This architecture ensures that at all levels the robots utilize coordinated behaviors, coordinated task execution and coordinated planning. Each robot essentially has these three layers and on an individual robot the layers can exchange information while on a robot-to-robot basis the synonymous layers (e.g. the executive layer) talk to each other. 3. The CAMPOUT architecture, designed by Huntsberger et al. [78], is an architecture that is able to autonomously adapt to the uncertainties of a dynamic environment. “CAMPOUT is a distributed control architecture based on a multi-agent behavior-based methodology, wherein higher-level functionality is composed by coordination of more basic behaviors under the downward task decomposition of a multi-agent planner. Basically CAMPOUT provides the infrastructure, tools and guidelines that consolidate a number of diverse techniques to allow the efficient use and integration of these components for meaningful interaction and operation”. CAMPOUT is comprised of five different architectural mechanisms including, behavior representation, behavior composition, behavior coordination, group coordination and communication behaviors The above architectures are but a few of the complex architectures that have. 18.

(40) 2.1. MULTI-ROBOT SYSTEMS. been developed strictly for multi-robot systems, other architectures have been proposed and presented in [23; 56; 151; 159; 171].. 2.1.3. Main Problems among a Group of Robots. Communication plays an important role in multi-robot systems and can increase their capacity and effectiveness, however, is one of the main problems among a group of autonomous robots due to its complexity and dynamism as it depends on environmental conditions as the interaction between themselves. The amount of information that is exchanged at a time can vary from one problem to another and consequently increases the degree of coordination depending on system complexity [85]. Communication in a multi-robot system is the ability possessed by members of the system to transmit and receive information between them, in a system of multiple robots can be two types of communication [163]: intentional or direct, in which used dedicated devices to ensure an effective communication. In this first type, the messages have a defined receiver which it always get the information, that is, communication is transmitted and received via some sort protocol or language as a medium. The second type is the non-intentional or indirect, in which information is transmitted by environmental changes or by visible state of the agents, also known as stigmery. In this type of communication there is no specific receptor for messages, that means, agents can leave marks and trails that can convey information to other agents that will recognize these changes in the environment. Several investigations have been directed to the problem of communication and information flow between multiple robots. Different works focusing on this problem and have been presented taking into account limited communication [30; 58; 110] and recently there has been an increased interest about the selfemergence of a common lexicon in robot teams [96; 101; 104].. 19.

(41) 2.1. MULTI-ROBOT SYSTEMS. 2.1.4. Coordination Schemes: Cooperative and Competitive. Cooperative and competitive methods provide a means of coordinating behavioral response for conflict resolution and offer an alternative to competitive. The coordination can be viewed as a competition among behaviors; this type of competitive strategy can be performed in a variety of ways. Generally, a coordination function (serving as an arbiter) selects a single behavioral response. The function can take the form of either a prioritization network (in which a strict behavioral dominance hierarchy exists) or an action-selection method (in which, on the basis of sensor information, only the most active behavior is selected). As we have previously commented a MRS has several advantages over a single robot, however, there are many problems that need to be considered in a dynamic environment, for example, multiple moving objects, various obstacles, team members, among others. All this makes more difficult to achieve coordination between robots. Currently one of the main interests of the international community is design strategies for communication and coordination for MRS, which allow robots to modify their behavior to cope with the environmental changes or actions performed by other robots, in order to obtain cooperative behavior that allows them to achieve a common goal. In previous works [128; 129] we have presented a control architecture to achieve cooperative and competitive behaviors in a MRS in an unknown environment. It has established a surveillance scenario where there are two teams of robots: the red robots must patrol and detect the blue robots in an office-like environment (see Fig. 2.2). The objective of red robots is to work coordinately in order to catch the blue robots (cooperative), meanwhile the goal of blue robots is to avoid be caught by any member of red robots (competitive). In another work [130], we have proposed two alignment strategies for self reconfiguration of modular mobile robots by means of cooperative behaviors. The strategies are based on a modular robot system [5; 168] using mobile reconfigurations and simulated to accomplish the task. The cooperative behaviors allow robots to modify their behavior to cope with environmental changes or actions performed by other robots, in order to obtain cooperative behavior that allows. 20.

(42) 2.2. FIELDS OF APPLICATION. Figure 2.2: Multi-robot system them to achieve a common goal. According to the results experimental obtained in both works, the coordination of multi-robot systems in dynamic environments require a well-structured control architecture, and to achieve collaborative behavior between members of a system, it needs a combination of behaviors associated with each robot. The results demonstrates that implementing the cooperative behaviors to both robots is the fastest way to achieve self-alignment for the docking process.. 2.2. Fields of Application. Currently, there are many fields of application that require the use of a group of robots, able to exhibit it more versatile behavior and flexibly to a great variety of situations. For this reason, research on MRS has increased and is being a field much studied by several researchers. Traditionally, robotics applications [124] were focused mainly in the industrial sector (e.g. welding, assembly, processing, workpiece handling, cutting materials by robot), where the main objective was the massive automation in services for increase productivity, flexibility, quality, and above all, to improve security to reduce the risk of people in dangerous tasks. In the past two decades, application fields of robotics has been extended to other sectors [17] some examples are: robots for construction [6; 71] (e.g. buildings, tunnels, roads, bridges, walls; domestic service robots [29; 137] (e.g. vacuum cleaners, lawn-mowing, window cleaning, pool funds, tanks, tubes and pipes; de-. 21.

(43) 2.2. FIELDS OF APPLICATION. fense robots, rescue and safety [94; 112; 138] (e.g. rescuing victims, deactivators mines, fire fighting and explosives, surveillance and security systems; assistive robots [70; 97] (e.g. helps disabled wheelchair, operational rehabilitation robots, wearable rehabilitation robots and other welfare functions; robots in medicine [118; 134] (e.g. diagnostic methods, surgical and interventional robotics, robotassisted recovery and rehabilitation, behavioral therapy, personalized care for special-needs populations. At the present, applications of multi-robot systems span a broad spectrum of areas, including human-unreachable environments, such as space, underwater, and rescue; challenging domains, such as construction and teams of unmanned aerial vehicles; and adversarial domains, such as robot soccer. Various specific tasks are addressed, e.g., foraging and coverage of a given area, multi-target observation, object pushing and transportation, exploration and flocking [158]. There are several areas of research that currently being explored in the field of MRS, focusing mainly on issues of coordination, cooperation, communication, localization, resource conflicts, architectures, among others. These applications, require more than one robot to complete a specific task and are needed to control the robots simultaneously to ensure synchronicity between them. MRS have numerous applications and can involve different fields of robotics, for example, industrial, military and service, or research and study of biological systems, and they can greatly affect different types of missions, for example, exploration, box pushing, the military operation, navigation in an unstructured environment, traffic control, entertainment, simulations of biological systems (see Fig. ??). In some industrial applications, for example, concern the possibility to move large objects that hardly a single robot can be sufficiently powerful to push alone a object and it can not enable to apply forces in all generalized directions. Therefore, a multi-robot solution can be useful for share the needed power among multiple robots.. 2.2.1. Cooperative Manipulation. Some tasks can require transporting objects (see Fig. 2.3), to achieve that a team of robots cooperate to carry a large object in an environment containing. 22.

(44) 2.2. FIELDS OF APPLICATION. static and dynamic obstacles, it is not an easy task. Different works about MRS have been discussed and presented in the literature to achieve this type of mission generally called Box-Pushing Mission, for example, in [107] are presented some experimental results of box pushing using two legged robots, in the works [59; 160; 166] have presented different methods for the problem of transporting objects by multiple mobile robots, the work in [152] presents an approach to carry a deformable object by means of two mobile robots with manipulators on board. Some have addressed the aerial transport of objects using cables [57; 109] and in [77] have proposed a solution to the problem box-pushing with multiple autonomous robotic fish in an underwater environment. Finally, others have taken inspiration from ant societies [9; 89].. Figure 2.3: Box-Pushing Mission [59; 107; 160; 166] and group of mobile robots designed to work cooperatively lifting columns (http://birg.epfl.ch/ page28710.html). 2.2.2. Unstructured Environments. The exploration in unknown environments (see Fig. 2.4) with a team of mobile robots is another kind of application which have been extensively studied in the literature in many forms. To achieve this mission in a cooperative way, all the robots must be coordinated to explorer different parts of the environment with. 23.

(45) 2.2. FIELDS OF APPLICATION. goal to cover the whole environment in less time than a single robot. Several authors proposed multi-robot exploration strategies based on market principles, in which robots place bids on subtasks of the exploration attempt and does not require a central agent, in [142] have proposed a distributed bidding algorithm for multiple robots in exploration tasks and addresses the problem caused by the limited communication range. The work in [18] presents an approach to explore an unstructured environment that has been implemented on real robots for different environments. Another approaches for coordination of multiple robots using market- based approach were proposed in [141; 172]. Figure 2.4: Exploration in unstructured environments. (a) The Mars exploration rovers, Spirit and Opportunity, with a manipulator arm in front, (b) a conceptual drawing for robotic rescue of Hubble space telescope, (c) The Pathfinder rover, Sojourner and (d) Rocky 4.. 2.2.3. Formation Control. Research on formation control involves a collection of decision making agents with limited processing capabilities, locally sensed information, and limited interagent communications, all seeking to achieve a collective objective (see Fig. 2.5). In the recent years, there is growing interest in distributed control due to its many advantages such as energy saving, scalable property and robustness [92;. 24.

(46) 2.2. FIELDS OF APPLICATION. 95]. Formation control is one of the most studied problems in MRS and many researchers start working on the consensus based formation control [20; 36; 54; 127; 135; 164]. In the leader-follower approach, each robot is assigned a leader from which it must maintain certain constraints [27; 52; 67; 147; 148; 162].. Figure 2.5: Formation Control. (a) Flying in Formation Takes Aircraft Farther, Dylan Ashe (http://www.popsci.com/). In (b) shows image of Vicon cameras overlooking a group of Khepera III robots. 3 cameras shown, 8 cameras total [98]. 2.2.4. Biologically-Inspired. The field of application in multi-robot systems has increased in recent years, several investigations have focused on the applications of biological inspiration as they provide fascinating examples of functional collective behavior [119; 136], characterized by rapid changes, high uncertainty, indefinite richness, and limited availability of information. These examples have been useful to study and apply these findings to the design of multi-robot systems. The first works inspired in the behavior of social insects (e.g., ants, bees, birds and fishes) in relation to the study of group behavior have been presented in [91; 106; 120]. Most bio-inspired robots are designed for specific tasks and for different environments (see Fig. 2.6), in order to cope with uncertain situations and react quickly to unforeseen changes in the environment. Pfeifer et al. [125] have presented a study about self-organization, embodiment and biologically inspired robotics.. 25.

(47) 2.3. PREVIOUS AND RELATED WORK. Figure 2.6: Bio-inspired robotics. 2.3. Previous and Related Work. Several researchers have addressed the problem of coordination in MRS, currently, there are several studies that focus mainly on the coordination of a set of robots using different techniques, in order to solve a specific problem. In the following subsections, we review some potential trends of research articles related with the coordination of multi-agent systems, swarm robots and multi-robot systems. In particular, we focus on previous and related work to coordination in MRS, reviewing some of the approaches to coordination that employ formal methods.. 2.3.1. Formal Methods in Relation to Coordination. In the last decade, there has been an increasing interest in systems comprised of several autonomous mobile robots, and as a result, there has been a substantial amount of development in this field; several researchers have studied the use of formal methods for the coordination and control of MRS. These works focus mainly on the coordination of a set of robots using different techniques, in order to solve a specific problem. With regard to the optimal tasks assignment problem, a brief review of some potential trends of research articles related to the coordination of multi-agent systems, swarm robots and multi-robot systems will. 26.

(48) 2.3. PREVIOUS AND RELATED WORK. be presented here. The discussion is focused on the recent literature in the area coordination with multiple robots. 2.3.1.1. Multi-Agent Systems. Researches in multi-agent systems about self-organization and emergence focus on naturally inspired approaches [62; 82] and socially-based approaches [72], have been studied and experimented with several mechanisms leading to self organization [10]. Price and Tiño suggest a number of strategies to address problems of task allocation in multi-agent systems, based on the principle of self-organization of social insects through the mathematical model developed by Bonabeau. They make a comparison of decentralized algorithms (FIFO and Greedy) to measure and evaluate the effectiveness of each strategy to process the mail and at the same time minimizing the number of changes [126]. The problem that has been considered for these algorithms of adaptation is a variation of the mail retrieval proposal by Bonabeau. Shang and Wang [140] have applied a similar problem of congestion of public resources in multi-agent systems: the famous “El Farol” bar problem in which a population of N agents have to self-coordinated respect to attendance at a place with limited capacity C, much lower than N. This strategy provides a simple mechanism for a large collection of decentralized decision makers to solve a complex congestion problem. Agassounon and Martinoli [2] have proposed a system for collecting objects, similar to response threshold completely deterministic, that is, when the stimulus exceeds a threshold determined immediately begins the execution of the task. In this case, it uses the time to find an object as stimulus to decide whether a robot should to run the task or rest. 2.3.1.2. Swarm Robots. In general, researchers in swarm robotics are inspired by the decentralized selforganizing biological systems and collective behavior of social insects in particular. 27.

(49) 2.3. PREVIOUS AND RELATED WORK. [68]. Swarm robotics is a novel approach to robotics which tries to circumvent problems with classical, monolithic robots like inflexibility and individual complexity by applying the principles of swarm intelligence to the field of robotics [44]. Typically these systems are composed of robots that, at the individual level, have relatively limited capacity to solve the task and limited knowledge about their environment. The general paradigm is often referred to as swarm intelligence [16; 47; 61]. Baglietto et al. have presented a coordination approach to swarm robots both navigation and task allocation based on RFID (Radio Frequency Identification, RFID). RFID devices are distributed a priori in the environment by building a navigation chart; each RFID device contains navigation instructions that allow the robots to run the routes from one place to another. Robots cannot communicate with each other, but may do so indirectly by writing and reading RFID devices. To perform the distributed task allocation algorithm defines an auction, where the central server takes work to be undertaken by a team of robots, analyzes and decides the number of robots, then robots are informed about the new tasks The allocation is the result of negotiations that each robot makes its own. Similarly using RFID devices to communicate, leaving registration messages between them, for example, messages and records assignments and out of zones. The system has been implemented in Player/Stage and navigation algorithm has been tested in MATLAB [7]. In the study by Yang et al. [167] have proposed a foraging mission in swarm robots, using mechanisms of response threshold with a nondeterministic selection of the task to be performed. Experiments have been implemented in TeamBots. 2.3.1.3. Multi-Robot Systems. One of the most popular approaches based on auction market mechanisms for the coordination of multi-robot systems was introduced by Dias and Stentz [33] in 2000. They consider that in multi-robot systems based on auctions, the robots are designed as agents of their own interests operating in a virtual economy. The. 28.

(50) 2.3. PREVIOUS AND RELATED WORK. tasks are assigned to the robots through the auction market mechanisms, for each task the complete robot generates some income that are reflected in the form of virtual money for providing a service to the team. However, when executing a task, the robot consumes resources such as fuel or network bandwidth, therefore, requires some expenses to pay for the resources used to complete the task. In 2004 [34] Dias has developed a coordination mechanism called Traderbots, which is designed to inherit the effectiveness and flexibility of a market economy. In this approach, were made some improvements in relation to the estimated costs to improve the efficiency of the team, then, in 2006 [86] this mechanism was applied in teams of harvesting to search treasure in an unknown environments. Shiroma and Campos have proposed a framework for coordination and distribution of tasks between a set of heterogeneous mobile robots called CoMutaR (Coalition formation based on Multi-tasking robots), allowing the robots to perform multiple tasks same time. It is based on the Contract Net Protocol to form coalitions concurrent through actions, use an auction process of a single round. They considered two specific experiments: (1) that two robots cooperate to push a box and (2) that a set of three tasks are performed by two robots [143]. Gerkey and Matarić have proposed an auction method for multi-robot coordination in their MURDOCH system [64]. A variant of the Contract Net Protocol, MURDOCH produces a distributed approximation to a global optimum of resource usage. The work basically shows the effectiveness of distributed negotiation mechanisms such as MURDOCH for coordinating physical multi-robot systems. In most of the previous work, the communication between robots is assumed to be perfect, which makes their algorithms unable to handle unexpected, occasional communication link breakdowns. Song et al. have proposed a Distributed Bidirectional Auction algorithm for multi-robot systems coordination. A task is divided into n sub-tasks, a robot can only run a sub-task, the allocation of sub-tasks is decided by both the auctioneer and bidder; the auctioneer chooses the pre-winners ordering the prices of offer, while the bidders chosen all tasks that pre-won the sub-task which has the lowest. 29.

(51) 2.3. PREVIOUS AND RELATED WORK. price. After the first round, the sub-tasks that were not chosen by any bidder enters a second round of auction depending on the initial price auction, this process is repeated until all sub-tasks have been completed [146]. In [93] Lim et al. have presented an architecture based on the auction market for the cooperation of a team of robots. On this platform, each team of robots is controlled by a respective MRS Client program and communicating through ZigBee Wireless Personal Area Network (WPAN). Each WPAN is assigned with a different identity (ID) so the data security of communicated information be preserved. A client program that acts as a buyer is used to deliver the tasks for users in the market. Then, a server program of tasks coordination is used to compare the buyers’ demand matches the supply from sellers. These programs are based on client/server architecture and are connected through Local Area Network (LAN) using Transmission Control Protocol (TCP) and Internet Protocol (IP).. 30.

(52)

(53) Part II Setting the Problem. 32.

(54) Chapter 3 Problem Description Most of the fundamental ideas of science are essentially simple, and may, as a rule, be expressed in a language comprehensible to everyone. Albert Einstein. SUMMARY: This chapter defines the problem statement proposed in this thesis. Section 3.1 establishes the idea of research, describing the issues related to the problem of coordinating a team of robots, mainly, the task assignment between them. Section 3.2 presents a formal description of the problem. Section 3.3 shows the experimental scenario established to carry out the experiments with different decentralized approaches. Finally, section 3.4 details the description of the proposed solution to the previously defined problems.. 33.

(55) 3.1. PROBLEM STATEMENT. 3.1. Problem Statement. The topics of research on MRS have been studied by several researchers of the scientific community due to the complexity of these systems. The problem of coordinating a team of robots involves a series of challenges that going beyond manipulation, modeling and navigation of the robot, that means, to accomplish a large task which can be divided into smaller parallel subtasks where a group works on an individual subtask. For example, in some works presented by Zlot et al. have been demonstrated the ability to handle task decomposition and loosely coordinated tasks using market-based techniques [173; 174]. Task assignment implies determining the order in which sub-tasks should be completed, groups that must meet each sub-task, and robots that should belong to which groups. Once the task assignment is completed, robots should be found with their new groups. In addition, groups should be able to communicate with other groups to ensure that the overall task is completed. In MRS, optimal task/job allocation or assignment is an active research problem, in which several central or global allocation methods have been proposed [79]. The probabilistic approaches have been used to solve major challenges of mobile robotics, getting some new and innovative solutions to important problems such as navigation, localization, tracking and robot control. This approach could be applied to the problem of coordinating multiple robots to the self-election of heterogeneous specialized tasks.. 3.2. Formal description of the problem. The optimal multi-task selection problem in multi-robot systems can be formally defined as follows: • “Let L = {l1 (t), l2 (t), ..., lJ (t)} be the different specialized tasks. Each lj ∈ L has a number of j jobs or pending loads where J = {j1 , j2 , ..., jK }. Let R = {r1 , r2 , ..., rN } be the set of N heterogeneous mobile robots. We made several assumptions concerning the problem description mentioned above; we have supposed that all members R = {r1 , r2 , ...rN } are able to participate in any jobs or pending loads lj ”.. 34.