México a
INSTITUTO TECNOLÓGICO Y DE ESTUDIOS SUPERIORES DE MONTERREY
PRESENTE.-Por medio de la presente hago constar que soy autor y titular de la obra denominada
, en los sucesivo LA OBRA, en virtud de lo cual autorizo a el Instituto Tecnológico y de Estudios Superiores de Monterrey (EL INSTITUTO) para que efectúe la divulgación, publicación, comunicación pública, distribución, distribución pública y reproducción, así como la digitalización de la misma, con fines académicos o propios al objeto de EL INSTITUTO, dentro del círculo de la comunidad del Tecnológico de Monterrey.
El Instituto se compromete a respetar en todo momento mi autoría y a otorgarme el crédito correspondiente en todas las actividades mencionadas anteriormente de la obra.
Automated Biped Gait Generetion for Robot Nao in Webots
Simulator Using a BioInspired ApproachEdición Única
Title
Automated Biped Gait Generetion for Robot Nao in
Webots Simulator Using a BioInspired ApproachEdición
Única
Authors
Ernesto Hiram Torres Domíguez
Affiliation
Tecnológico de Monterrey, Campus Monterrey
Issue Date
20110501
Item type
Tesis
Rights
Open Access
Downloaded
18Jan2017 14:12:14
Automated Biped Gait Generation for robot Nao in
Webots simulator using a Bio-inspired approach
by
Ernesto Hiram Torres Domínguez, B.Sc.
Thesis
Submitted to the Gradúate Program in the
División of Mechatronics and Information Technologies
in partial fulfillment of the requirement for the
Master of Science
degree i n
Intelligent Systems
Instituto Tecnológico y de Estudios Superiores de Monterrey
Campus Monterrey
Automated Biped Gait Generation for robot Nao in
Webots simulator using a Bio-inspired approach
by
Ernesto Hiram Torres Domínguez, B.Sc.
Thesis
Submitted to the Gradúate Program in the
División of Mechatronics and Information Technologies
in partial fulfillment of the requirement for the
Master of Science
degree in
Intelligent Systems
Instituto Tecnológico y de Estudios Superiores de Monterrey
Campus Monterrey
Instituto Tecnológico y de Estudios Superiores de
Monterrey
Campus Monterrey
División of Mechatronics and Information Technologies
G r a d ú a t e Program
Thesis committee members recommend the present thesis of Ernesto Hiram Torres Domínguez to be accepted in partial fulfillment of the requirements for the Master
of Science degree in:
Intelligent Systems
Gradúate Program Director
Acknowledgements
This thesis has been possible with the help and support of many people. M y parents, Amador Torres and Margarita Domínguez, who have always supported me and who always have encourage me to never give up and follow my dreams. Because of them I could study in Monterrey, N . L . and followed my dreams of studying robotics.
Dr. Leonardo Garrido for all those meetings to discuss the research. For his advise and recommendations on my thesis and on papers. For taking an interest on RoboCup and being an enabler on tournaments and contests.
M y teachers, for transferring their knowledge and helping me when I had doubts. M y friends from the lab of robotics. For all those long discussions and recommen dations about my thesis. For the time they gave me listening to my ideas and helping me select a path when I had doubts about how to proceed. For those coffee breaks that helped me clear my mind.
M y friends outside the school. For listening to my "crazy" talk and helping me take breaks from my work. For remembering me to relax and have fun.
To Prof. Ijspeert for providing me the papers and works in oscillators made by his research team.
To C O N A C y T for the scholarship they gave me to continué with my studies.
E R N E S T O H I R A M T O R R E S D O M Í N G U E Z
Instituto Tecnológico y de Estudios Superiores de Monterrey
Automated Biped Gait Generation for robot Nao in
Webots simulator using a Bio-inspired approach
Ernesto Hiram Torres Domínguez, M.Sc.
Instituto Tecnológico y de Estudios Superiores de Monterrey, 2011
Thesis advisor: Leonardo Garrido Luna, P h . D .
This thesis presents a solution to the biped locomotion problem. The robot used for the experiments is robot Nao by Aldebaran Robotics and it is simulated in Webots mo bile robot simulator. We provide a methodology for automated biped gait generation for a robot. The robot generates motions to move in its environment. The motions learned include a frontal walk and lateral left and right movement. Our method of solution does not requires the dynamic model of the robot, thus making this approach usable to other biped robots. For faster results the number of degrees of freedom is kept low, only six are used for generating a frontal locomotion. For lateral walk only four degrees of freedom are used.
The walking gait is generated using Central Pattern Generators with limitcycle oscil lators. We suggest the use of Central Pattern Generators (CPGs) based on limitcycle oscillators, together with Genetic Algorithms ( G A ) . The C P G generates trajectories that are followed by the robot's actuators, and it is optimized by the use of G A s to find the best synchronization. Our solution is generated automatically and the best results for the frontal walk allow the robot to walk twice as fast as the Aldebaran's webots walk and four times faster than the default walk in Robotstadium.
Contents
Acknowledgements v
Abstract vi
List of Tables x
List of Figures xi
Chapter 1 Introduction 1
1.1 Motivation 1 1.2 Problem Definition 2
1.3 Hypothesis and Research Questions 3
1.4 Objetives 3 1.5 Methodology • 4
1.6 Contributions 5 1.7 Organization 5
Chapter 2 Background Information and Related Work 7
2.1 RoboCup 7 2.1.1 Leagues and rules 8
2.2 Webots simulator and Robotstadium 9
2.2.1 Webots 9 2.2.2 Robotstadium 10
2.3 Robot Nao 10 2.4 Genetic Algorithms 11
2.5 A review on biped robots 13 2.5.1 Human anatomy, anatomical planes and walking cycle 13
2.5.2 Robot model 16 2.6 A review on C P G s 17
2.6.1 Coupled oscillators and synchronization 17
2.8.1 Oscillator Description 22 2.8.2 Other contributions 22
2.8.3 Applications 23
2.9 Summary 25
Chapter 3 T h e Walking Nao in a Simulated Environment 26
3.1 Controller Overview 26 3.2 Experiment Methodology 27 3.3 Preliminary Experiments 27
3.3.1 A r m synchronization 28 3.3.2 Simulator programming 29 3.3.3 Oscillator Selection 31
3.4 Frontal Walk 32 3.4.1 Number of required D o F 33
3.4.2 Connections 33 3.4.3 Genotype and Genetic Algorithm 35
3.4.4 Experimental Results 36
3.4.5 Additional considerations • 38
3.4.6 Comparative between "Borregos" team controller and our controller 39 3.5 Lateral Walk 42 3.5.1 Connections 42 3.5.2 Genotype and Genetic Algorithm 43 3.5.3 Lateral Gait Results 44 3.6 Result Discussion 46 3.7 Summary 47
Chapter 4 Conclusión and Future Work 49
4.1 Research Questions Revisited 49
4.2 Contributions 50
4.3 Future Work 51
A p p e n d i x A M o d e l N a o V 3 R 53
A p p e n d i x B Kinematics 54
A p p e n d i x C Webots and G A L i b 55
A p p e n d i x D Preliminary Experiments for the Physical Robot 56
D . l External Libraries 56 D.2 Communication 56
List of Tables
2.1 Nao's principal characteristics concerning height, weight, energy, degrees
of freedom, among others 11 2.2 Parameters required for the experiment on two coupled oscillators . . . 23
2.3 Mojon's results from the experiment on two coupled oscillators 24 2.4 Comparative table between previous work by Mojón and this thesis. . . 25
3.1 Parameters required for the arm synchronization experiment 29 3.2 Oscillators used for frontal locomotion. In the left the limitcycle and
the sinusoidal on the right 31 3.3 Comparative parameters for the limitcycle oscillator and sinusoidal os
cillator 32 3.4 Oscillator valúes. The valué range for a variable is displayed in each cell,
missing valúes are valúes not required 34
3.5 Genotype 35 3.6 Solution obtained for the C P G (Split in two for display purposes). . . . 37
3.7 The genotype proposed for the lateral gaits (The parameters are one
single strip, separated for display purposes) 43 3.8 Parameters required for the G A configuration 43 3.9 The genotype for left lateral gait with the numerical solutions obtained
(The parameters are one single strip for each genotype, separated for
display purposes) 45 3.10 The genotype for right lateral gait with the numerical solutions obtained
(The parameters are one single strip for each genotype, separated for
List of Figures
2.1 The online contest Robotstadium running in Webots simulator. The
image displays the two teams ready for a soccer match 11 2.2 Crossover between two individuáis. The ending valúes of each genotype
is switched at the cross point. Only one crosspoint is used 13 2.3 Sagital improvements. Starting with a compasslike walk with only one
DoF to later add the knees and finally the ankles to maximize propulsión 15 2.4 Frontal improvements. The focus in this plańe is the stability compen
sating the lateral movements 16 2.5 Transversal improvements. Helps change direction and orientation. . . . 16
2.6 The walking cycle, starts with the right heel making contact and in double support phase. Next, in single support the right foot and the left swinging. Ends when left foot lands to be in double support phase.
Showing only left full step 17 2.7 Limitcycle oscillator in phase space 23
3.1 The controller overview. It links the C P G with the robot in order to
genérate a gait. The G A optimize the parameters required by the C P G . 26
3.2 A r m synchronization experiment on shoulder joints 29 3.3 Fitness per Generation between the limitcycle and the sinusoidal os
cillators. Displaying the best individual from each experiment and the
average for 10 experiments of each oscillator 32 3.4 The Oscillator connections. A l l the connections are bidirectional. . . . 34
3.5 Comparative between the oscillations generated by the C P G (bottom)
and the real oscillations by the servomotors (top) 37 3.6 Fitness per Generation displaying the best individual and the average
for 10 experiments 38 3.7 Nao walking. Can be appreciated from topleft to bottomright 39
3.8 Race between the controllers. Top blue: Aldebaran's controller. Center
black: our controller. Bottom red: Robotstadium's default walk . . . . 40 3.9 Race between the controllers. Center black: our controller (r ¡ lAcm/s).
3.10 Race between the controllers. Center black: our controller (~ 18cm/s).
Bottom red: Borregos team controller ( « lQcm/s) 41
3.11 Frontal gait with acceleration. The speed increased allowing the robot
to walk faster, generating a sprint 41 3.12 The Oscillator connections for the lateral gaits. A l l the connections are
bidirectional 42 3.13 Right lateral gait. This gait only requires the ankles and hip rolls. . . . 44
3.14 Left lateral gait. The oscillations generated for the gait 45
3.15 Lateral walking in the right direction 45 3.16 Lateral walking in the left direction 46 3.17 Fitness per generation of left and right lateral gaits 46
A . l The servos valúes according to the model provided by cyberbotics. . . . 53
Chapter 1
Introduction
In October 1992 in Tokyo, a group of japanese researchers interested in artificial intelligence discussed grand challenge problems and selected the game of soccer as a way to promote science and technology. In June 1993, they decided to launch a soccer robot competition. In that time, it was tentatively named the Robot JLeague. The competition will allow the development of soccer robots to motivate the research in áreas such as robotics, artificial intelligence and multiagent systems. One month after the launch of the contest, researchers worldwide became interested in the project and was extended as an international joint project. Robot World Cup Initiative(RoboCup for short) [16] was born. RoboCup is the inspiration for this thesis.
This thesis presents a methodology for automated motion generation for a biped robot. The robot generates motions to move in its environment. The motions learned include a straight forward walk and lateral left and right movement.
The robot used as principal research tool is the robot Nao [1] by Aldebaran Robotics together with Webots simulator [5]. This research gives the Nao the ability to move and walk without the requirement of the dynamic model by using a bioinspired ap proach. We suggest the use of Central Pattern Generators (CPGs) based on limitcycle oscillators, together with Genetic Algorithms ( G A ) . The C P G generates trajectories that are followed by the robot's actuators, and it is optimized by the use of A G s to find the best synchronization.
1.1 Motivation
algorithms to favor the automatic learning for a robot to learn by trial and error, just as a human would learn something new. B y using A I , a robot could accomplish side steps, a rotational movement or even walking. A n algorithm that can be robust enough to be independent from a robot architecture is still unaccomplished, specially for biped robots in which the stability plays a critical role. A biped robot not only should genérate motion trajectories, it also has to learn not to fall. Our research interest is to genérate in automated manner the motions required for a robot to be able to move in its environment. This is why in this research a method for solving this problem is presented. A bioinspired approach with Central Pattern Generators (CPGs) and genetic algorithms (GA) is proposed. This methods have the advantage of being inspired by nature and their performance are shown in the evolution of the species. The G A , by using selection, crossover and mutation, generates a population of individuáis that evolve and become better at ac complishing a specific task, in this case, walking. We selected G A s for being very good at finding solutions when the evaluation is simple. The C P G s [14] are used to créate a controller that is equivalent in the way humans and animáis genérate locomotion. The C P G is the low level locomotion controller and generates the motion that a higher level controller requires, such as the interaction between the spinal cord and the brain. Both C P G and G A were used to genérate the controller in a bioinspired approach.
1.2 Problem Definition
For this problem, we created an algorithm and a controller that uses C P G s to genérate goalbased trajectories, not caring too much in having a exact and precise movement but instead in the walk itself. The focus is in its behavior. The C P G gives robustness to the walk and is able to genérate the periodic behaviors found in locomotion.
1.3 Hypothesis and Research Questions
Using a bioinspired approach using Central Pattern Generators (CPGs) and Ge netic Algorithms (GA), it is possible to créate an algorithm together with a controller to achieve the automatic generation of motions required for a robot to be able to move in its environment.
This research has a high number of options available and its correct selection will determine its full potential. Several questions remain and will be answered through the progress of this research:
• Are the limitcycle oscillators the best way to implement a C P G instead of puré sinusoidal oscillators? The sinusoidal oscillators may be simpler to implement and may have the same effect as the limitcycle oscillators. It is also possible that the automatic synchronization of limitcycle oscillators will give better results.
• Would it be necessary to use all the D o F of the robot Nao in order to achieve the best gait possible? The Nao has 21 DoF and adding an oscillator to each of the D o F will genérate a huge search space for the G A and it may not converge. Some D o F will contribute more than others to the locomotion.
• Would the solution obtained be better than the actual walk motion in "Borregos" Robotstadium team? The solution may be good, but its use in a competitive environment has to be determined. It may replace the existing motion used in the Robotstadium team and help it play better.
1.4 Objetives
This thesis covers several objectives in order to be fulfilled:
• Test and implementation of C P G s . In order to achieve a full controller the basic functionality has to be tested. To know what C P G s can do and what can not is a key aspect and it will give us a good starting point.
• Adapt the frontal C P G to genérate lateral gaits. A frontal walk is important for a robot, but so it is the lateral walk since it allows the robot to cover an opponent without losing track of the ball.
• Implement controller into actual "Borregos" team. The current Robotstadium team is missing a good walking gait, causing the robots to fall. The new controller should replace the existing locomotion controller.
1.5 Methodology
In order to have a path to follow, a methodology has been proposed as a guideline of the steps required to achieve a bioinspired controller that will be implemented in a Nao robot. The accomplishment of each step is fundamental, enabling the next one. Each completed step helps to achieve the objectives proposed.
1. Revise the different approaches for walking in biped robots to justify the bio inspired approach as the best way of solution. There are many approaches for biped locomotion and each of them has its advantages and disadvantages.
2. Select a proper oscillator to use in the central pattern generators. The selection of the oscillator is an important decisión since it is the basic unit for the C P G . A n incorrect oscillator may give more problems as the research progresses.
3. Test and implement a single oscillator to validate its properties. Ensure it can work in the C P G for locomotion generation.
4. Test and implement oscillator coupling. The correct interconnection of the C P G is very important and the oscillators should be able to genérate periodic behaviors when coupled.
5. Test oscillator coupling in Nao with simple movements. A test in the Nao will give the key starting point, by using them on the robot architecture we can make adjustments and ensure the proper functioning.
6. Decide the minimal degrees of freedom(DoF) to be used for a biped walk. The more the D o F , the larger is the search space and that will reflect in a longer convergence time for the algorithm. It is important to define which DoF are going to be used for the best movement and faster convergence.
8. Decide if its necessary to increase the number of D o F to achieve a better walk and do if may improve the controller. If a gait is achieved it may be possible to increase its speed and stability by adding extra DoFs.
9. A d d extra adaptations to aid the controller perform in a soccer. Adding lateral movements could be a good improvement and will allow the Robot to use the same C P G for different movements.
10. Implement results in the controller of "Borregos" team. Once tests have been conclusive it will be time to implement the full controller in the team.
1.6 Contribuí ions
Our research contributes with the following:
• A robust algorithm to genérate frontal and lateral gaits. The algorithm was used to genérate both frontal and lateral gaits using the same fitness function.
• Locomotion controller with C P G s with Small number of D o F . B y reducing the number of D o F used the search space is reduced, resulting with faster results.
• Adaptive gaits with frequency change in realtime. The gaits generated can be modified in realtime to increase its speed.
• Validation of a methodology for gait generation independent of the robot dynamic model. Using Central Pattern Generators together with Genetic Algorithms pro vide a methodology to genérate gaits.
1.7 Organization
This work is divided in four chapters:
Introduction. In this chapter the problem is introduced. In order to solve the prob
lem we present our hypothesis and our research questions to guide the research. Objectives are proposed and a methodology for solving the problem is presented. The organization includes a brief description on the chapters that will follow.
Background Information and Related Work. This chapter covers the simulation
T h e Walking Nao in a Simulation Environment. This chapter is divided in three sections covering the experiments of this research.
• Preliminary Experiments. It includes experiments regarding the oscillators and the simulation environment. This section presents the starting point of the research. It includes the oscillator selection experiment for comparing a limitcycle oscillator and sinusoidal oscillator.
• Frontal Walk. It includes the process of selection for the D o F that are used, the connections for oscillator coupling and the G A configuration. Research results are provided and are used as the foundation for generating lateral gaits. Includes other adaptations such as a sprint to cover large distances.
• Lateral Walk. Includes the adaptation of the algorithm to genérate lateral gaits in right and left direction. Research results are presented regarding the lateral gaits showing its adaptability to genérate more than just frontal gaits.
C o n c l u s i ó n and future work. General discussion of the results of the research and
Chapter 2
Background Information and Related Work
This chapter describes the concepts required for a better understanding of this thesis and the related work on the biped locomotion problem. We present the RoboCup initiative and how it was initiated. Our tools, Robotstadium and Webots simulator are described briefly to show their uses and how they intégrate with this research. The robot Nao, its characteristics and description are incorporated. We introduce the Genetic Algorithm as the optimization algorithm for this research. The next sections give reviews on the biped robots, Central Pattern Generators ( C P G ) and the walking types for readers that may be new to these topics. Finally, we present related work to the biped locomotion problem and work related to the same C P G used in this research.
2.1 RoboCup
RoboCup [16] is an initiative to increase the development of artificial intelligence and robotics. The main divisions of RoboCup are: RoboCup rescue, RoboCup soccer and RoboCup home. Our focus is in RoboCup soccer. The domain of soccer was se lected for being a complex sport in which many abilities are required in order to play. A soccer team will have a good performance by developing and implementing research on: autonomous agents, multiagent coordination, computer visión and robotics. In order to guide the research and motivate it, RoboCup initiative decided to créate a goal that is difficult enough but that can still be achieved. The visión of RoboCup is to have by 2050 a team of robots that not only will be able to play soccer, they will be able to defeat the world soccer champions.
Universities from all the world have supported the initiative and every year interna tional competitions take place in a guest country. RoboCup not only enforces the participation in these contests, but also creates symposiums and conferences to share the knowledge acquired among all participants. In addition, RoboCup also creates specific challenges to focus on certain problems in the soccer game, such as: dribbling, passing and coordination.
allow the robots perform in more limited environments, with the goal to play in a real soccer field with a real soccer ball. A n example of these updates is the switch from the fourlegged Aibo robots to the biped Nao.
2.1.1 Leagues and rules
There are several leagues for RoboCup Soccer, these leagues were created to help focus on hardware and software. Simulation leagues were created to focus on software and algorithm development, including 2D and 3D simulation. Humanoid league was created to help improve mainly the hardware but also encourage the software integration from simulated environments. In this league each team builds their own robots only limited by the máximum size of the category and having human proportions. Finally, the Standard Platform League(SPL) in which each team use identical physical robots. The competitive edge is the code used to program the robot.
2D and 3D simulation. In 2D simulation there two teams with 11 players, each
player is an autonomous software program called an agent. The field is two dimensional represented by a central server. The server knows all the information for all agents in the game. The game relies on the communication between an agent and the server. The agent receives information about its virtual sensors and must decide what action it should execute. Research in Multiagent systems is key in this type of simulation, since all agents are equally capable. 3D simulation evolved from 2D simulation to make the game even more complex. Started with a robot equivalent to the H O A P 2 by Fujitsu and changed to use Aldebaran Nao. The current game is played by two teams with no more than six players. Each robot has simulated sensors such as a camera, accelerometers, gyroscopes and forcé sensors. Humanoid. In this league humanlike robots play a soccer match. The common prob lems in this league are dynamic walk, visual perception of the ball, selflocalization and agent coordination. To solve this problems the robots include sensors to gen érate perceptions on its own and other robots states. The robots are custom made for playing soccer and are among the best autonomous robots available. There are three subcategories: Kidsize (30cm60cm), teensize (100120cm) and adultsize (greater than 130cm). In this category we made our first entrance in physical robots, participation in the Second Mexican Robotics Tournament.
S P L . This league is one of the favorites for all the researchers who want to avoid all
Sony A I B O and later changed to the humanoid Aldebaran Nao. For this category a special RoboCup edition was made sacrificing the hand and wrist articulations to reduce costs. Like in the other leagues each robot is autonomous and does not require a human to control it. As part of the Autonomous Agent research group, our research interest is in the S P L league and we are currently developing a team to participate in RoboCup 2012.
2.2 Webots simulator and Robotstadium
A simulation environment helps to test experiments without risking the physical integrity of the robot. B y using a simulator, algorithms that require a huge number of iterations can be implemented and controlled. Webots is a simulation environment created specifically for research in robotics and holds the Robotstadium contest to allow users to submit their code and compete agains other users or research groups.
2.2.1 Webots
Webots [5] is a development environment used to model, program and also sim úlate mobile robots. In Webots, the user may design their own robot as complex as they want to. The test environment can also be programmed and each object can have its own properties such as: shape, color, texture, friction, etc. It includes a builtin I D E for robot controller programming, and can also be programmed by thirdparty development environments. Webots is used by a large number of universities for its flexible and robust simulation. It is well documented and has been maintained for over 10 years.Other advantages of Webots include multiplatform support, it can be used in W i n dows, Linux and MacOS operating systems. In addition, it supports programming with: Java, C / C + + , U R B I , Python or Matlab.
Programming both is fundamental for artificial intelligence algorithms requiring the múltiple reset of the environment to the same initial conditions everytime.
2.2.2 Robotstadium
Robotstadium [6] is an online contest based on Webots simulation using Alde baran Nao robot as the model. The contest is organized by Cyberbotics and it is an open contest free of charge. The contest is played according to the S P L league rules and is a way to test the research made in robotics and A I .
Currently, it is played with three players on each team, one goalkeeper and two field players. A match consists in two half times, each of them with a duration of 10 min utes. After the first halftime the teams switch sides. If a game is tied at the end of the second half, a penalty kick shootout will take place. For a user/team to particípate it is only required to download the latest versión of We bots and run the robotstadium world. A controller can be programmed for the robots in any of the programming languages supported by Webots. When finished the con troller is uploaded to the website and they will do the rest. Matches will take place and each team will score points based on winning games. Downloading the game matches is an option too. Based on the performance the team may update the controller and submit fixes. Having Robotstadium competitive environment, our team created controllers for the field players and the goalkeeper in order to participate. Their controllers are based on our research experiments and in the last season 2010 we reached the hall of fame, scoring a second place. Robotstadium uses a model of the Nao called NaoV3R. It is based completely in the physical Nao and emulates its sensors and properties. Nao capabilities and characteris tics will be explained in the next section. Robotstadium running on Webots simulator can be appreciated in Fig. 2.1.
2.3 Robot Nao
Nao is a high performance bipedal robot designed and manufactured by Alde baran Robotics. Nao carne to replace Sony A I B O for the RoboCup competition in the Standard Platform League. Nao has been used in S P L since 2008.
Figure 2.1: The online contest Robotstadium running in Webots simulator. The image displays the two teams ready for a soccer match.
Table 2.1: Nao's principal characteristics concerning height, weight, energy, degrees of freedom, among others.
characteristics as the academic edition, but it doesn't have actuated hands, having only 21 degrees of freedom available. The academic edition also includes a Choreographe license for high level programming based on a graphical interface. A summary on its characteristics are available in Table 2.1.
For additional information, Appendix A includes the robots servos and its descrip tion. Appendix B displays the Nao Kinematics.
2.4 Genetic Algorithms
Genetic Algorithms (GAs) are search method inspired in nature, they are based on natural selection and genetics [8]. A G A encodes the variables of a search problem
[image:27.612.170.437.71.219.2] [image:27.612.125.486.324.450.2]in a imite length string. The code corresponds to an alphabet of a certain cardinality. The codes representing the variables are often referred as the chromosomes or the geno types. This type of algorithms is used to solve complex problems such as the traveling salesman. among others.
In order for a G A to identify the good solutions it implements an objective function of ten referred as fitness function. It represents a function that measures the performance of an individual. The criteria for the objective function can be a mathematical model. The G A is based in the concept of a population, a group of individuáis or candidate solutions. It is a parameter of the algorithm and affects the performance. A small population may lead to quick solutions but may be premature and the solutions may not be the best. If the population is large, it may waste computational time.
There are a certain steps followed by the G A to obtain a solution to a problem:
1. Initialization. The initial population is generated with random valúes for the genotype. 2. Evaluation. Using the fitness function the individuáis are evaluated and assigned a valué. 3. Selection. The genotypes with the higher valúes are selected for the crossover pool. This step is required to ensure the best individuáis are preferred to the worse individuáis. 4. Crossover. This step is also known as recombination. Two individuáis are selected and they are recombined to créate two new genotypes that are possibly better solutions. 5. Mutation. This step acts over each of the newly created individuáis. It involves one or more modifications of an individual. 6. Replacement. The initial population is replaced with the new individuáis.
The G A continúes its execution with the iteration of steps 2 to 6 until a criteria for stopping the algorithm is met. Reaching a number of generations or using convergence are commonly used to stop the G A .
crossovers. A crossover point is selected randomly in the case of onepoint. It is illus trated in Fig.2.2.
Once the new population is created there are several approaches to replace the oíd
Figure 2.2: Crossover between two individuáis. The ending valúes of each genotype is switched at the cross point. Only one crosspoint is used.
population. One approach is to delete all and replace it completely. Another approach is Elitism in which the best individual of the oíd generation is preserved and inserted in the new population.
2.5 A review on biped robots
Our human body design is really useful, it gives us the ability to walk upright leaving free our superior limbs for grabbing things. These abilities are available to us because of our biped walk. We live in a human world, and the design of all the constructions and objects have only human users in mind. Then. It seems natural to build robots with our characteristics so they can navigate and use as much as the things we already have without having to adapt anything. Also, by recreating robots with our resemblance, we are able to understand our body better and be able to apply the knowledge in other áreas of interest such as rehabilitation and sports.
Research on biped robots is a complex topic, by using only two extremities to walk, the balance plays a crucial role in walking and has to be considered when designing gaits. The next subsections will briefly mention important concepts that will be used through the rest of the thesis.
2.5.1 Human anatomy, anatomical planes and walking cycle
H u m a n anatomy In [4], they mention information about the human anatomy regarding its appli cation to biped robotics. The human lower limbs consists in 44 bones, in which three of them are the main support: the fémur, the tibia and the fíbula. The rest are part of the foot and they aid with extra support and ńexibility. In addition to the bones, there are 46 skeletal muscles, but only the larger muscles cause a greater effect in the propulsión movement. The muscles works in pairs, having one contract and another one stretch. The architecture given by the bones and muscles allows our leg to be composed by six main degrees of freedom: three D o F at the hip articulation, one D o F at the knee and two D o F at the ankle. Another two D o F are located at the feet, but are often dismissed to simplify the model. Anatomical planes The walking cycle can be analyzed from different perspectives, these perspectives are called anatomical planes. There are three anatomical planes of the body. The frontal plańe separates the body into back and front portions. The sagittal plańe trav eling from the top to the bottom of the body dividing into left and right proportions. Finally, the transverse plańe dividing the body into lower and upper body parts. Humans have a large number of degrees of freedom due to their muscle and bone struc ture, and it can not be implemented completely in robotics because of the increment of the number of D o F . The increment in D o F will increase the possible trajectories and the ńexibility of the locomotion is amplified, but it is not currently cost effective, since the processing time also increases. These limitations require to make a simplified model that can achieve walking, but have a reduced complexity. The structure is analyzed from the three perspectives previously mentioned. It is mainly in the sagittal plańe (Fig. 2.3) where the walking movement takes place. The simpler structure includes only two hip articulations that allows both legs to swing back and forth creating the impulse required for walking, this makes a compasslike walking. A better model includes a knee articulation on each leg. Finally, another two articulations can be added to correspond to ankles that will help in the forward propulsión. In the frontal plańe (Fig. 2.4), the principal concern is the lateral stability of walking. The stability depends on the center of gravity in relation to the contact point of the stance leg with the ground. Hip and ankle modifications are added to the model to allow balance correction and stability improvement. B y keeping the feet parallel to the floor a better adherence to the ground is guaranteed, resulting in a better stability and a bigger área of support.
Figure 2.3: Sagital improvements. Starting with a compasslike walk writh only one
DoF to later add the knees and finally the ankles to maximize propulsión
to five more articulations on the transversal plańe. Adding an articulation for trunk rotation can compénsate for for the movement of the balancing leg. If instead of adding it at the center of the trunk, two articulations are added it will allow the change of direction during walking. Finally, extra foot articulation will help oriéntate the foot to maximize the ground support or to even pivot on the grounded foot.
Walking cycle
Walking can be considered a cycle when the speed at which the movement is executed is constant. As discussed in [4], walking is a succession of grounding phases and balance. It is made a convention that the cycle starts when the contact is made with the right heel, followed by a left step and then a right step.
The walking cycle can be observed from the sagittal plańe to analyze the way the legs move and créate a forward propulsión. The phase in which both feet are grounded is called double support phase. Also the phase in which only one foot is on the ground is called monopodal phase, and that makes the other foot be in balancing phase.
[image:31.612.122.511.86.285.2]Figure 2.4: Frontal improvements. The focus in this plańe is the stability compensating the lateral movements.
the ground. The final step is left foot in monopodal phase while the right foot travels to make impact on the ground with the heel. Walking cycle repeats the times it is required.
It is important to mention that the phase of double support disappears when human walking is faster than 2.1 m/s.
2.5.2 Robot model
As mentioned in previous sections, the human ability to walk can not be easily transferred to a robot. The robot performing a human walk would require a high
[image:32.612.126.518.566.643.2]Figure 2.6: The walking cycle, starts with the right heel making contact and in double support phase. Next, in single support the right foot and the left swinging. Ends when left foot lands to be in double support phase. Showing only left full step.
number of D o F to move as ńuent as a human being. To simplify this problem most ap proaches only consider the lower part of the body, modeling only the legs and ignoring the behavior of the trunk and arms. This model as simple as it can be, it still requires at least 12 D o F to replicate the human legs. This would be for each leg: 3 D o F in the pelvis, 1 D o F for the knees, 2 D o F for the ankles.
2.6 A review on CPGs
The justification for this approach relies on the existence of oscillators in nature. Studies show that in vertébrate and invertebrate living organisms the brain is only the high level controller. The real controller exists in the central nervous system, formed by a group of neural networks that produce oscillatory outputs to the limbs. The brain then just requests for launching a walk, or run and the nervous system executes the locomotion. It has been proven the the existence this type of arrangement in the human too and can be read in detail in [7].
2.6.1 Coupled oscillators and synchronization
[image:33.612.137.475.94.217.2]The inventor of the pendulum clock, Dutch physicist Christiaan Huygens was the first one to notice the oscillator coupling. His discovery was due to observing how two pen dulum clocks on the same wall with different initial position will tend to swing in a perfect harmony. From this. he suspected they were affecting each other somehow. He continued a series of experiments explained in detail in [17]. A l l this experimentation created a subbranch in mathematics: the theory of coupled oscillators. Coupled oscillators can be found in the nature in a great number of ways, even though they appear more in living things, insulinsecreting cells, heart cells used for pacemaker, running, neural networks in the brain and spinal cord, among others. When using oscillators in biological systems, they not only have a characteristic period, but have also a characteristic amplitude. This special attribute allows the biological oscillators to return to their former path even after perturbations. This type of os cillators are known as limitcycle oscillators. They incorpórate a way of damping big oscillations and also a energy source that can raise the oscillations that are too small. The synchronization is the result of phase locking. In the case of two oscillator cou pling there are two possible results, synchrony which is a phase different of zero and asynchrony, a phase different of one half. If three oscillators are coupled instead of two, 4 possible patterns can be obtained. A l l in synchrony, having their phases shifted by onethird, two oscillators in synchrony while the other moves in unrelated way just with the same period and twooscillators moving with one half difference and the third one moving twice as fast as its neighbors. Examples of this types of patterns are a kangaroo, with each of its legs as an oscillator in which they are perfectly synchronized to allow the jumping. A human walk, focusing only on the foot are an example of a twooscillator system behaving in a asynchronous way.
2.7 Current approaches for walking
Many research has been made about biped walk in the past years, and can be classified according the type of walk used. There are two types of walk, static and dynamic.
Static walking. A static walk is considered to be stable at any moment, meaning that
the center of gravity of the robot is always located in the zone of stability called the polygon of support. Viewed from the sagittal plańe, during single support phase the center of gravity should be between the two extremities of the stance foot and during the double support phase it should be between the tip of the front foot and the end of the back foot. This type of walk is slow but considered not to difńcult to implement.
Dynamic walking. In this type of walk, the center of gravity is almost never in the
be able to react according to the velocity and inertial forces generated by the movement. It may be more difficult to implement, but the increase of speed movement that provides, makes this type of walk a good research topic.
Both dynamic and static walks are used in robots, each of them has its advantages and disadvantages, for one being faster and harder to obtain and the other slower, but easier to genérate. Current approaches focus mainly on dynamic walk. There are several classes based on the approach used to obtain a solution.
Trajectory based
This type of approach performs all the calculations needed to obtain the correct trajectory of every joint at every moment. It requires a good amount of computa tional power. The most widely used method is the Z M P (Zeromovement point). From [19], Z M P is the point on the waking ground surface at which the horizontal components of the resultant moment generated by active forces and moments action on human/humanoid links are equal to zero. A n d the second one, Z M P is the point in the floor at which the moments around x and y axis generated by reaction forcé and moment are zero. Both interpretations point out certain aspects of the Z M P and still leave the basic notion unchanged. It is also stated that Z M P should not be confused with the Center of Pressure(CoP).
This type of approach can provide excellent stability, but it requires a dynamic model of the robot in order to follow a well stablished methodology. The dynamic models depends on which robot is being used. It also requires online computation.
Good results can be obtained by using this method. In [18], Z M P was used to créate an engine for an omnidirectional walk. The calculation of a Z M P using the full dynamic of the robot is computationally intensive and not suitable for competitions, to avoid this they present a simplification with the inverted pendulum mode:
(2.1)
In where x is the position of the center of mass (CoM), zh is the constant height
of the C o M from the ground and g is the acceleration of the gravity (9.81^). After modeling the robot, a preview controller generates dynamically balanced center of mass trajectories to compute the desired position of the feet and thus deciding the direction of the walk.
Learning
in this type of algorithms is that they do not require to have a dynamic model of the robot, but instead, try to learn from experience on a goalbased algorithm. Learning algorithms are classified based on their outcome. The drawback of machine learning is the time consumption.
In [3], an extensión of the classic Policy Gradient algorithms is used, taking into ac count the parameter relevance allows for better results when only few experiments are available. Their focus is on curvilineal biped walks.
For their step parameters they used:
• The duration of a step.
• The ratio between single support and the total step duration.
For the foot trajectories, they take into account:
• The total length of the step.
• The abscissa of the swinging foot at the beginning of the single support phase.
• The portion of distance covered during one double stance phase.
• The stance foot altitude.
• Máximum height from the ground reached by the swinging foot.
The objective of the learning was to learn the optimal parameter vector that ensures the best performance. The experiments were performed in Webots and on real Naos. Their work confirmed the execution of walking gaits even after a limited number of experiments.
Based on heuristics
Since the dynamic model of a robot can be a very difficult task. B y using heuristics a correct trajectory can be calculated with out having to calcúlate the equations needed. Genetic algorithms can be used for this type of approach. It is often used together with another method since the search space for the variables required for walking can be very large. Optimization algorithms are heavily used.
In [13], a genetic algorithm was used together with partial Fourier series (PFS) in RoboCup 3D. The G A were used to genérate offline parameters that define the gait. Based on the premise that human movements are inherently periodic and repeat the same set of steps an oscillator was proposed:
where N is the number of frequencies, C is the offset, ^4„=I..AT are amplitudes, T is the
period and 4>n=\..N
a r e phases. For each joint an oscillator is used. Each joint used
generates 4 parameters, the more joints the more parameters will be required for the system. Using symmetry, they focus only on one side of the robot, reducing the number of parameters by half.
In the G A , the fitness function was:
(2.3)
where c4a« is the distance to the ball and 0 is the average of the oscillations of the
C o M . They tried to minimize their fitness function. They achieved good results having a walk forward velocity of 0.51m/s in RoboCup 3D simulation league. Bio-inspired Instead of creating new methods, the bioinspired approach creates algorithms based on the observation of the living creatures. It tries to understand the existing models in the nature that have been evolving for millions of years and use them to solve complex problems. The advantage of this approach is the independence from the model of the robot, and the focus on behaviors. A n example, It is not required to have a perfectly followed trajectory in order to walk, but instead it is goaloriented to the walk itself.
The main component in a bioinspired approach are the central pattern generators (CPGs) [10] , which are rhythm generated networks that control the locomotion. Since C P G s are used to créate the movement trajectory, is often common to use reflexes as control to cióse the loop and improve stability and balance. In bioinspired models, genetic algorithms are often the algorithm of choice for parameter optimization since it is based on evolution.
In [12], the C P G network consists of the Matsuoka neuron model, and is introduced to realize the locomotion of a bipedal robot, they also use genetic algorithms in several steps to find out large number of parameters according to structure of C P G network. In [11], an oscillator is proposed for a salamander robot to genérate the transition between swimming and walking. This oscillator was used in [2] to genérate locomotion for modular robots and in [15] to genérate trajectories for the biped robot Hoap 2. It is also the oscillator selected for this thesis.
2.8 Previous Work on Limit-cycle Oscillators
2.8.1 Oscillator Description
The oscillator used for the C P G was proposed by Professor A . Ijspeert from Biorob [11]. It is a nonlinear oscillator and has the interesting property of having the limit cycle behave like a sinusoidal signal with an amplitude \f~E~ and a period 2-KT.
(2.4)
TX = v (2.5)
The variable x represents position and v represents velocity. Variables r,a and E are
positive constants. B y modifying T, the period of the oscillator can be manipulated
and changing a changes the system speed convergence. The energy of the system is
represented as E. For oscillator coupling:
T±Í = Vi (2.7)
Where and bid represent how oscillator j influences oscillator i. The last sumatory
allows for sensory input.
A previous work by Mojón [15] suggests a modification on the oscillator to allow the connection strength to be independent from the energy of the oscillators. This is achieved by normalizing the connections:
The last equations represent the final model of the oscillator used in this work. One oscillator per D o F was used. The oscillator can be appreciated in Fig. 2.7.
2.8.2 Other contributions
In [15], a study is presented on a system based on two coupled oscillators. The objective of the study was to understand their behavior to later use the results for a better understanding on a full system. The experiment objective was to understand the parameters required to forcé a phase difference between two coupled oscillators. For the experiment the following parameters in the Table 2.2 were required.
Fixed valúes were used for E, xO, r and a. The valúes of the connection weights were
(2.6)
(2.8)
Figure 2.7: Limitcycle oscillator in phase space
Table 2.2: Parameters required for the experiment on two coupled oscillators
changed iteratively to find the relation between them. The Table 2.3 shows the results of the experiment.
This contribution was used in order to genérate a 7r difference between the two legs on
the C P G and be able to reduce the number of required parameters by half.
2.8.3 Applications
There are three main applications using the oscillator presented in the previous sections: A salamander, modular robots and biped robots. The first use, is proposed by Professor Ijspeert for a salamander to créate a transition from swimming to walking. The modular robots focused on generating the best gaits for a set of robots. Finally, the last application is on biped robots and using the oscillator to créate a fullbody C P G .
Salamander robot. In [11], they presented a model of the salamander's locomotion
controller based on nonlinear oscillators. It features a salamander that is able to
[image:39.612.185.412.58.200.2] [image:39.612.143.467.280.384.2]Table 2.3: Mojon's results from the experiment on two coupled oscillators
use the same C P G for swimming and walking, making the change when it was on ground or water. A double chain of oscillators were connected in the spinal cord to allow the undulations that favored movement.
M o d u l a r robots. This work was made focusing on modular robots locomotion. It
used the same oscillator proposed in [11]. It features four types of modular robots:
a wheel (having four modules connected in a loop), a Caterpillar (connected as a
snake), a t e t r a p o d (similar to a starfish) and a quadruped.
One oscillator per D o F was used and the number of parameters varied for each robot. For the solution, several optimization methods were proposed. A random search, genetic algorithms, particle swarm optimization (PSO) and simulated annealing. The algorithms tried to maximize the distance covered by the robot. The best method of solution was P S O . The oscillator source code was used as base and modified and adapted for our specific needs.
B i p e d robot. The aplication on bipeds, uses an oscillator in each of the D o F of the
robot, the biped robot used is Hoap 2. The joints used are: two for the ankles, one for the knee, three for the hip, one for the back, two for the shoulder and one for the elbow. Symmetry was used to reduce the number of parameters. A Ge netic algorithm was used for parameter optimization and used a complex fitness function:
where distReached is the straight line distance traveled, standingTime is the
time elapsed before the robot falls, totalTime is the total simulation time and stabTime the time needed by the C P G to stabilize itself.
[image:40.612.166.473.109.175.2]of the good performance of the algorithm. The main differences between this work and this thesis are displayed in Table 2.4.
Table 2.4: Comparative table between previous work by Mojón and this thesis.
2.9 Summary
In this chapter, we presented the RoboCup initiative and how it motivates re searchers around the world to make advances in robotics and artificial intelligence. The main leagues and the leagues in which we have participated were introduced. Also, the simulation environment and the robot used for the research.
Chapter 3
The Walking Nao in a Simulated Environment
This chapter covers the following: the controller overview, the preliminary experi ments, the frontal walk and the lateral walk. The controller overview explains the main components and the task they carry on. In the preliminary experiments we cover tests and validation of the simulation environment and we select the main oscillator. In the frontal and lateral walk sections we explain the process to genérate the gaits.
3.1 Controller Overview
Our controller integrates three main components. The Genetic Algorithm ( G A ) , the Central Pattern Generator ( C P G ) and the robot. The gait is generated by their interconnection. In Fig. 3.1 the components are illustrated.
First is the G A , its main function is to optimize the parameters required by the C P G .
Figure 3.1: The controller overview. It links the C P G with the robot in order to genérate a gait. The G A optimize the parameters required by the C P G .
[image:42.612.202.435.440.574.2]a resting position and all the motors are at the position zero. The positions of the oscillators are passed to the robot and it moves the servomotors to the new position. Each simulation step the C P G computes a new position and the robot updates the servomotor position valúes. A gait is generated.
3.2 Experiment Methodology
Our research includes three main experiments: The oscillator selection, the frontal walk and the lateral walk. The oscillator selection compares two different oscillators in order to select the best. In the frontal and lateral walk we use the selected oscillator to créate gaits. For each experiment a methodology was followed:
• Decide the number of D o F used for the experiment. in this step we decided which joints were used for the C P G . This decisión affects the connections between the DoF and the type of propulsión that can be achieved. Also affects the number of required parameters and the search space for the variables.
• Créate the connections between joints. Once the joints have been selected we proceed to control the position of each joint using an oscillator. Each oscillator may or may not be connected to other oscillators, depending on the experiment. In the case of the limitcycle oscillator the general idea is to have all of the oscillators connected to créate a network. In the sinusoidal, it is only necessary to place an oscillator in each joint, but since the sinusoidal oscillator does not have a mechanism for coupling no connections are necessary.
• Configure the G A . The parameters that the G A will optimize are now fixed according the joints used and the type of the oscillator. The genotype is formed by the concatenation of each of the parameters. The fitness function is not fixed and several experiments are required to decide the best one. The parameters for the G A are based on previous experience. We tried to be consistent with the G A parameters and used the same valúes for all the experiments, only changing the population to be twice the size of the parameters.
• Obtain and analyze results. The experiment will run according to the setup we decided and the results are analyzed.
3.3 Preliminary Experiments
robot is very complex and we had to confirm all the parts making up the experiment functioned by their own before interconnecting them:
• The first test was about the oscillators. The related work validated the synchro nization between oscillators, but they were validated with other robots different from the Nao. We confirmed they could be correctly implemented in the webots environment and assure they could couple and synchronize to a desired phase. Also verify the Nao can follow the oscillations and have a good performance.
• A t first we generated our own genetic algorithm and later decided to change it to G A l i b [20] to avoid investing time in adding functionality to the algorithm that may or may not be used.
• Robotstadium contest provides a sample code for Nao programming in a sim ulation environment, we improved and modified the code to meet our specific needs.
• Since we wanted to automatize the process of finding solutions, a supervisor needed to be programmed. The supervisor was in charge of controlling the robot position.
The last section includes experiments for the comparison between a puré sinusoidal oscillator and the limitcycle oscillator used in this thesis. The best oscillator was selected for the full experiment of the frontal and lateral gait generation.
3.3.1 Arm synchronization
Table 3.1: Parameters required for the arm synchronization experiment.
(a) S h o u l d e r s w i t h 0 p h a s e dif (b) S h o u l d e r s w i t h ir p h a s e dif
ference ference
Figure 3.2: A r m synchronization experiment on shoulder joints shown in (a) and (b)
3.3.2 Simulator programming
After the oscillator coupling was successful, we decided to continué with the ex periment programming. We implement a G A for the optimization and several program ming and adjustments were done to the simulation world, the robot controller and the supervisor controller.
T h e genetic algorithm
A genetic algorithm is commonly used for optimization problems. They involve a population of individuáis that become better with each iteration. The G A s follow four simple steps each iteration: genérate a population, the selection of the best individuáis, crossover between the best individuáis and mutation of the new generated population. If the G A is executed enough iterations and it is properly configured, it finds the solution to an optimization problem.
[image:45.612.178.445.106.400.2]