• No se han encontrado resultados

Stereovision Feedback and Fuzzy Control for Autonomous Robot Navigation Edición Única

N/A
N/A
Protected

Academic year: 2020

Share "Stereovision Feedback and Fuzzy Control for Autonomous Robot Navigation Edición Única"

Copied!
83
0
0

Texto completo

(1)INSTITUTO TECNOLOGICO Y D E ESTUDIOS SUPERIORES D E M O N T E R R E Y CAMPUS. MONTERREY. DIVISION D E M E C A T R O N I C A Y T E C N O L O G I A S D E L A I N F O R M A C I O N. STEREOVISIÓN F E E D B A C K A N D F U Z Z Y. CONTROL. FOR AUTONOMOUS ROBOT NAVIGATION. THESIS PRESENTED AS A PARTIAL. REQUISITE. TO OBTAIN T H E DEGREE:. M A S T E R O F SCIENCE IN A U T O M A T I O N. BY: A R I S T E O HERNÁNDEZ. M O N T E R R E Y , N.L.. MARTÍNEZ. D E C E M B E R O F 2010. 1.

(2) INSTITUTO TECNOLOGICO Y D E ESTUDIOS SUPERIORES D E M O N T E R R E Y CAMPUS. MONTERREY. DIVISION D E M E C A T R O N I C A Y T E C N O L O G I A S D E L A I N F O R M A C I O N. STEREOVISION F E E D B A C K A N D FUZZY. CONTROL. FOR AUTONOMOUS ROBOT NAVIGATION. THESIS PRESENTED AS A PARTIAL. REQUISITE. TO OBTAIN T H E DEGREE:. M A S T E R O F SCIENCE IN A U T O M A T I O N. BY: A R I S T E O HERNÁNDEZ. M O N T E R R E Y , N.L.. MARÍNEZ. D E C E M B E R O F 2010. 2.

(3) I N S T I T U T O TECNOLÓGICO Y D E E S T U D I O S SUPERIORES D E M O N T E R R E Y CAMPUS MONTERREY DIVISIÓN D E MECATRÓNICA Y TECNOLOGÍAS D E L A INFORMACIÓN The members of the thesis committee recommend that this thesis written by B.Sc.Aristeo Hernández Martínez, is accepted as a partial requisite to obtain the academic degree of: M A S T E R O F SCIENCE IN A U T O M A T I O N Thesis Committee:. Dr. Sergio Sedas Gersey Committe Member. M.Sc. Miguel de Jesús Ramírez Cadena Committe Member. APPROVED. Director de las Maestrías de Electrónica y Automatización de D M T I. M O N T E R R E Y , N.L.. D E C E M B E R O F 2010. 3.

(4) Contents 1. 2. Introduction. 10. 1.1. Motivation. 10. 1.2. P r o b l e m Statement and Context. 10. 1.3. Solution Proposal. 11. 1.3.1. Stereovision. 11. 1.3.2. Fuzzy Control. 11. 1.4. M a i n Contributions. 11. 1.5. Thesis Structure. 11. Theoretical Framework. 13. 2.1. Stereo V i s i o n. 13. 2.2. R o a d Detection a n d Image Processing. 15. 2.3. Fuzzy Control. 16. 2.3.1. D e f i n i t i o n of t h e F u z z y Sets. 16. 2.3.2. O p e r a t i o n s w i t h F u z z y Sets. 16. 2.3.3. Fuzzy Relations. 17. 2.3.4. Operations w i t h Fuzzy Relations. 17. 2.3.5. Approximate Reasoning. 17. 2.3.6. Inference R u l e s. 18. 2.3.7. T h e I F - T H E N Rules. 18. 2.3.8. Fuzzification ( F M ). 19. 2.3.9. Inference M a c h i n e. 19. 2.3.10 D e f u z z i f i c a t i o n ( D M ) 2.4. 3. 20. R e a l - T i m e Systems. 20. 2.4.1. 21. R e a l - T i m e Scheduling Policies. State of A r t. 22. 3.1. R o a d D e t e c t i o n w i t h Stereo V i s i o n. 22. 3.1.1. Horizon Estimation. 23. 3.1.2. Obstacle Detection. 24. 3.1.3. 3D V i e w. 24. 3.2. F u z z y C o n t r o l i n A u t o n o m o u s M o b i l e Vehicles. 24. 3.3. C o m p a r i s o n of W o r k s. 26. 4Proposal 4.1. 28. Stereovision system. 28. 4.1.1. Image acquisition. 28. 4.1.2. Image processing. 29. 4.2. Fuzzy Controller. 31. 4.3. Software P l a t a f o r m. 36. 4.4. Hardware. 36. 4.

(5) 5. Experiments and Results. 37. 5.1. Hardware. 37. 5.1.1. 38. Actuators. 5.1.2. Sensors. 40. 5.1.3. Processing. 40. 5.2. Image Processing a n d R o a d Detection. 41. 5.3. Fuzzy Controller. 49. 5.4. Software I m p l e m e n t a t i o n. 52. 6. Conclusions. 53. 7. Further Work. 54. References. 55. A. H a r d w a r e Specifications. 61. A.1. StereoCamera. 61. A.2. Servomotors. 63. A.3. Servo controller c a r d. 64. A.4. P o w e r M o d u l e of t h e S p e e d C o n t r o l. 66. A.5. Arduino Microcontroller. 67. A. 6. Laptop. 69. B. C. Program Description. 70. B. 1. Image Processing. 70. B. 2. Fuzzy Controller. 79. D e s c r i p t i o n of E x p e r i m e n t s. 80. C. 1. Road Detection. 80. C.2. Distance Calculation. 80. C.3. Object Detection. 82. C.4. Controllability. 82. 5.

(6) List of Figures 1. C o m p o n e n t s of a n a r t i f i c i a l v i s i o n s y s t e m. 13. 2. C o m p o n e n t s of software for a r t i f i c i a l v i s i o n s y s t e m s. 14. 3. Stereo V i s i o n T r i a n g u l a t i o n. 15. 4. C o m p o n e n t s of a u t o n o m o u s r o a d f o l l o w i n g. 23. 5. D e t e c t i o n of t h e h o r i z o n l i n e a c c o r d i n g t o [3]. 23. 6. Integral solution. 28. 7. F l o w c h a r t of t h e i m a g e p r o c e s s i n g. 29. 8. Points detected a n d calculated i n the image processing. 31. 9. F l o w c h a r t of t h e f u z z y c o n t r o l l e r. 33. 10. B l o c k d i a g r a m of t h e f u z z y l o g i c c o n t r o l l e r. 34. 11. I n p u t m e m b e r s h i p f u n c t i o n s for e r r o r i n p. 34. 12. I n p u t m e m b e r s h i p f u n c t i o n s for e r r o r i n 9. 35. 13. O u t p u t membership functions. 35. 14. B a s i c s t r u c t u r e of t h e m o b i l e r o b o t .. 37. 15. Servomotor attached to the steering system. 38. 16. S y s t e m t h a t r e g u l a t e s t h e s p e e d of t h e r o b o t. 38. 17. Servomotor controller card configuration. 39. 18. B l o c k D i a g r a m of t h e S p e e d C o n t r o l. 39. 19. B l o c k D i a g r a m of t h e S t e e r i n g S y s t e m. 40. 20. Stereocamera.. 40. 21. F r o n t P a n e l of t h e v e h i c l e o n a s t r a i g h t l i n e w i t h p o s i t i v e R h o .. 41. 22. F r o n t P a n e l of t h e v e h i c l e o n a s t r a i g h t l i n e w i t h n e g a t i v e R h o. 41. 23. F r o n t P a n e l of t h e v e h i c l e o n a c u r v e w i t h n e g a t i v e T h e t a. 42. 24. F r o n t P a n e l of t h e v e h i c l e o n a c u r v e w i t h p o s i t i v e T h e t a. 42. 25. R e s u l t s of t h e s i m u l a t i o n. 45. 26. I m a g e w h e n n o o b s t a c l e is d e t e c t e d. 46. 27. D e t e c t i o n of o b s t a c l e i n p o i n t a. 47. 28. D e t e c t i o n of o b s t a c l e i n p o i n t b. 47. 29. D e t e c t i o n of o b s t a c l e i n p o i n t c. 48. 30. D e t e c t i o n of o b s t a c l e over t h e t r a c k i n p o i n t b. 48. 31. M e m b e r s h i p f u n c t i o n s of t h e i n p u t 9. 49. 32. M e m b e r s h i p f u n c t i o n s of t h e i n p u t p. 50. 33. M e m b e r s h i p f u n c t i o n s of t h e o u t p u t. 50. 34. M a n i p u l a t i o n , error a n d r h o i n car trajectory. 51. 35. F l o w c h a r t of t h e software s o l u t i o n. 52. 36. Webcam.. 61. 37. D i m e n s i o n s of t h e w e b c a m .. 62. 38. Servomotor HS-422.. 63. 39. Servo C o n t r o l l e r S S C - 3 2 .. 64. 40. P o w e r m o d u l e for t h e s p e e d c o n t r o l. 66. 41. F r o n t v i e w of t h e A r d u i n o U n o. 67. 42. B a c k view from the A r d u i n o U n o .. 68. 6.

(7) 43. Dell Inspiron 300m.. 69. 44. E d g e d e t e c t i o n s e c t i o n i n front p a n e l. 71. 45. T h r e s h o l d s e c t i o n i n front p a n e l. 71. 46. E q u a l i z a t i o n s e c t i o n i n front p a n e l. 72. 47. S t e r e o v i s i o n i n front p a n e l. 72. 48. C o n f i g u r a t i o n of s t e r e o c a m e r a i n front p a n e l. 73. 49. F r o n t p a n e l of t h e p r o g r a m. 74. 50. Configuring image acquisition. 75. 51. Equalization and threshold. 75. 52. Edge detection. 76. 53. O v e r l a y s a n d c a l c u l a t i o n of p a r a m e t e r s. 77. 54. Distance calculation. 77. 55. Block diagram. 78. 56. B l o c k d i a g r a m of t h e f u z z y c o n t r o l l e r. 79. 57. E x p e r i m e n t of d i s t a n c e c a l c u l a t i o n. 81. 58. E x p e r i m e n t of d i s t a n c e c a l c u l a t i o n. 81. 59. E x p e r i m e n t of d i s t a n c e c a l c u l a t i o n. 82. 60. Track. 83. 7.

(8) List of Tables 1. C o m p a r i s o n of t h e different w o r k s i n recent years. 26. 2. Set of r u l e s of t h e f u z z y c o n t r o l l e r .. 32. 3. E x p e r i m e n t s carried out. 37. 4. E d g e d e t e c t i o n d a t a of t h e left i m a g e. 44. 5. E d g e d e t e c t i o n d a t a of t h e r i g h t i m a g e. 45. 6. C o m p a r i s o n of r e a l a n d c a l c u l a t e d d i s t a n c e s. 46. 7. Rule Base.. 49. 8. S u m of S q u a r e d E r r o r. 50. 9. Experiments. 80. 10. Distances calculated. 81. 8.

(9) Part I. T h i s thesis presents t h e p r o j e c t of a n a u t o n o m o u s v e h i c l e t h a t follows a t r a j e c t o r y b y d e t e c t i n g t h e r o a d t h r o u g h s t e r e o v i s i o n p r o c e s s i n g a n d i t is c o n t r o l l e d w i t h t h e use of f u z z y l o g i c .. T h e v i s i o n p r o c e s s i n g i m p l e m e n t s edge d e t e c t i o n b y a n a l y z i n g lines of. pixels. T h e vision processing determines the error i n the actual position a n d the fuzzy c o n t r o l moves t h e s t e e r i n g of t h e c a r a n d slows d o w n t h e v e l o c i t y i f n e e d e d . i m p l e m e n t e d i n a mobile robot to determine current results.. 9. It was.

(10) 1. Introduction. T h e m a i n o b j e c t i v e of t h i s thesis is t h e i m p l e m e n t a t i o n of a n a u t o n o m o u s r o b o t t h a t is c a p a b l e of f o l l o w i n g a r o a d w i t h t h e use of a s t e r e o c a m e r a .. T h e use of v i s i o n i n 3 D. is a t r e n d i n m a n y v i s i o n a p p l i c a t i o n s , a n d we w o u l d l i k e t o a d o p t i t for. autonomous. vehicle navigation. R o a d f o l l o w i n g r e q u i r e s t w o c r u c i a l steps: of t h e s p e e d a n d s t e e r i n g of t h e v e h i c l e .. the road recognition a n d the control. T o r e a c h c o n t r o l l a b i l i t y i n these t y p e. of. r o b o t s i t is n e c e s s a r y t o d e t e c t t h e r o a d a n d m a k e t h e i m a g e p r o c e s s i n g t o c a l c u l a t e t h e m a n i p u l a t i o n s n e e d e d t o keep t h e r o b o t i n t h e d e s i r e d p o s i t i o n a n d p a t h . It c a n b e s a i d t h a t v i s i o n s y s t e m s are a v e r y i m p o r t a n t f a c t o r i n t h e. development. of a u t o n o m o u s vehicles b e c a u s e t h e y are t h e sensors of t h e r o b o t ; therefore, t h e i m a g e p r o c e s s i n g needs t o be d o n e o n t i m e i n o r d e r t o d e t e c t t h e r o a d a n d let t h e e n t i r e s y s t e m t o be c o n t r o l l a b l e . T h r o u g h t h i s t h e s i s , v a r i o u s t e c h n i q u e s i n different areas s u c h as r o a d d e t e c t i o n , i m a g e p r o c e s s i n g a n d f u z z y c o n t r o l t h e o r y w i l l be e x a m i n e d i n o r d e r t o achieve t h e objective.. 1.1. Motivation. A u t o n o m o u s m o b i l e r o b o t s w i t h i m a g e p r o c e s s i n g u s i n g c o m p u t e r s are a c h a l l e n g e for t h e c o n t r o l s y s t e m due t o t h e d e m a n d s i n c o m p u t a t i o n a l c a p a c i t y a n d also t h e a b i l i t y t o do t h a t p r o c e s s i n g as fast as p o s s i b l e t o a l l o w c o n t r o l l a b i l i t y of t h e v e h i c l e . tonomous. Au¬. r o a d f o l l o w i n g has b e e n r e s e a r c h e d t h r o u g h recent years b e c a u s e i t c o u l d. p r e v e n t a c c i d e n t s i n r e a l life s i t u a t i o n s a n d i t is a t r e n d i n a u t o n o m o u s v e h i c l e develop¬ m e n t . I n g e n e r a l , a u t o n o m o u s n a v i g a t i o n is a n i n t e r e s t i n g a r e a t o r e s e a r c h b e c a u s e i t demands real mechatronics: electronics, mechanics, computer p r o g r a m m i n g a n d control engineering.. 1.2. Problem Statement and Context. A u t o n o m o u s mobile robotics builds physical systems that can move purposefully a n d w i t h o u t h u m a n i n t e r v e n t i o n i n u n m o d i f i e d e n v i r o m e n t s a n d t h e d e v e l o p m e n t of tech¬ n i q u e s for a u t o n o m o u s. n a v i g a t i o n c o n s t i t u t e s one of t h e m a j o r t r e n s i n t h e c u r r e n t. r e s e a r c h o n r o b o t i c s . T h i s t r e n d is m o t i v a t e d b y t h e c u r r e n t gap b e t w e e n t h e a v a i l a b l e technology a n d the new application demands.. O n one h a n d , t h e t e c h n i q u e s. i n current industrial robots lack the ability to provide. flexibility. employed. a n d autonomy; on the. o t h e r h a n d , t h e r e is a c l e a r e m e r g i n g m a r k e t for t r u l y a u t o n o m o u s. robots.. Possible. a p p l i c a t i o n s i n c l u d e i n t e l l i g e n t service r o b o t s for offices, h o s p i t a l s , a n d f a c t o r y. floors,. m a i n t e n a n c e r o b o t s for i n a c c e s s i b l e areas, d o m e s t i c r o b o t s for c l e a n i n g o r e n t e r t a n m e n t , o r a u t o n o m o u s vehicles t o h e l p t h e d i s a b l e d a n d t h e ederly. T h e p r o b l e m t o be s o l v e d b y t h i s thesis is t h e n e e d of a n a u t o n o m o u s v e h i c l e c a p a b l e of f o l l o w i n g a r o a d a u t o n o m o u s l y ; therefore, t h e m a i n c o m p o n e n t s. for r o a d t r a v e l i n g. n e e d t o be c o n t r o l l e d : s p e e d a n d s t e e r i n g . I n o r d e r t o solve t h i s p r o b l e m , i n d i v i d u a l. 10.

(11) s i t u a t i o n s have t o b e c o n s i d e r e d , s u c h as: w a y s of d e t e c t i n g t h e r o a d a n d / o r o b s t a c l e s , t y p e s of c o n t r o l s t o be u s e d a n d t h e i m p l e m e n t a t i o n of software a n d h a r d w a r e .. 1.3. Solution Proposal. I n o r d e r t o solve t h e p r o b l e m , i t has b e e n d i v i d e d i n t w o m a i n areas: s t e r e o v i s i o n a n d fuzzy control. 1.3.1. Stereovision. T o detect t h e r o a d a n d t h e o b s t a c l e s , s t e r e o v i s i o n has b e e n p r o p o s e d t o be t h e sensor of t h e v e h i c l e b e c a u s e i t does n o t o n l y detects o b j e c t s i n t h e i m a g e , b u t also measures distances i n the frame. 1.3.2. Fuzzy Control. B e c a u s e t h e c o n t r o l a l g o r i t h m is also a f u n d a m e n t a l p a r t i n r o a d f o l l o w i n g for a s m o o t h r i d e a n d t o r e d u c e t h e e r r o r i n p o s i t i o n , f u z z y l o g i c is p r e t e n d e d t o be u s e d as a n e m u l a t o r of h u m a n e x p e r t i s e , b e c a u s e i t e x t r a c t s k n o w l e d g e of e x p e r t p e o p l e a n d w o r k s well w i t h imprecise data.. 1.4. M a i n Contributions. T h e m a i n c o n t r i b u t i o n of t h i s t h e s i s is t h e a l g o r i t h m of s t e r e o v i s i o n i m p l e m e n t e d t o r o a d f o l l o w i n g . T h i s a l g o r i t h m , a l t h o u g h s i m p l e , is p r a c t i c a l for t h e o b j e c t i v e n e e d. because. i t a l l o w s a fast p r o c e s s i n g of i m a g e s , w h i c h is a v e r y i m p o r t a n t for t h e c o n t r o l l a b i l i t y of t h e a u t o n o m o u s v e h i c l e .. T h i s thesis is also t h e base for f u r t h e r w o r k , i n o r d e r t o. e x p a n d t h e a p p l i c a t i o n s a n d f u n t i o n a l i t y of t h i s s t u d y .. 1.5. Thesis Structure. T h i s thesis shows t h e w o r k d o n e i n o r d e r t o solve t h e p r o b l e m . T h e thesis is s t r u c t u r e d as f o l l o w s , I n c h a p t e r t w o , t h e t h e o r e t i c a l f r a m e w o r k is p r e s e n t e d , a s u m m a r y of c o n c e p t s t h a t are n e e d e d t o u n d e r s t a n d w h a t is d o n e i n n e x t c h a p t e r s . I n c h a p t e r t h r e e , t h e state of a r t is p r e s e n t e d , w h e r e s i m i l a r w o r k s are p r e s e n t e d , w h i c h c a n b e t h e base for t h i s t h e s i s . I n c h a p t e r f o u r , t h e p r o p o s a l for t h e s o l u t i o n is p r e s e n t e d , i t i n c l u d e s t h e h y p o t h e s i s of h o w t h e o b j e c t i v e c a n be c o m p l e t e d as best as p o s s i b l e as far as we are c o n c e r n e d . I n c h a p t e r five, t h e e x p e r i m e n t s w i t h t h e a u t o n o m o u s m o b i l e r o b o t are s h o w n ac¬ c o r d i n g t o t h e o b j e c t i v e s a n d also c o m p a r a t i v e s w i t h s i m i l a r w o r k s . T h e r e s u l t s of t h i s e x p e r i m e n t s is s h o w n as w e l l . C h a p t e r 6 presents t h e c o n c l u s i o n s of t h i s s t u d y . C h a p t e r 7 proposes further w o r k to be done w i t h this thesis.. 11.

(12) T h e s e c t i o n of a p p e n d i x e s. is d i v i d e d i n t h r e e :. hardware specifications,. d e s c r i p t i o n a n d t h e d e s c r i p t i o n of t h e e x p e r i m e n t s c a r r i e d o u t .. 12. program.

(13) 2. Theoretical Framework. T h i s c h a p t e r shows t h e c o n t e x t over w h i c h t h i s thesis is b a s e d o n ; i n o r d e r t o u n d e r s t a n d t h e o b j e c t i v e of t h i s w o r k , t h e t h e o r e t i c a l f u n d a m e n t s have t o be s t u d i e d . It is i m p o r t a n t t o m e n t i o n t h e bases of r o a d f o l l o w i n g for a u t o n o m o u s v e h i c l e n a v i g a t i o n w h i c h for t h i s thesis are b a s i c a l l y d i v i d e d i n : stereo v i s i o n , r o a d d e t e c t i o n a n d i m a g e p r o c e s s i n g , f u z z y control a n d real-time systems. T h r o u g h t h i s c h a p t e r , t h e c o n c e p t s of f u z z y c o n t r o l are s t u d i e d t o u n d e r s t a n d t h e c o n t r o l l a b i l i t y of t h e a u t o n o m o u s v e h i c l e .. S t e r e o v i s i o n is also a f u n d a m e n t a l p a r t of. t h e p r o j e c t , w h i c h i n t h i s case, is t h e sensor t o d e t e c t t h e r o a d a n d t h e b o u n d a r i e s of t h e car; a n d i n o r d e r t o sense, a n i m a g e p r o c e s s i n g of t h e v i s i o n s y s t e m has t o b e d o n e .. 2.1. Stereo V i s i o n. A c c o r d i n g t o [25], a n a r t i f i c i a l v i s i o n s y s t e m has: •. C a m e r a , c a p t u r e s t h e i m a g e s a n d t r a s m i t t h e m as e l e c t r i c s i g n a l s , f o l l o w i n g rules of e x p l o r a t i o n .. •. Interface, i t a d a p t s t h e e l e c t r i c signals p r o d u c e d b y t h e c a m e r a t o t h e c o m p u t e r .. •. S o f t w a r e , i t a l l o w s t o a n a l y z e t h e scenes a n d generates t h e c o m m a n d s. for t h e. robot control autonomously a n d i n real t i m e (Artificial Intelligence). F i g u r e 1 shows t h e c o m p o n e n t s of a n a r t i f i c i a l v i s i o n s y s t e m .. F i g u r e 1: C o m p o n e n t s of a n a r t i f i c i a l v i s i o n s y s t e m I n t h e software p a r t , t h r e e c o n s e c u t i v e phases c a n be d i s t i n g u i s h e d : •. S e l e c t i o n , of t h e u s e f u l a n d i n d i s p e n s a b l e i n f o r m a t i o n , b e c a u s e i t is a l m o s t i m posible to take into account a l l the i n f o r m a t i o n the camera provides.. •. I n t e r p r e t a t i o n , of t h e scene i n a c o n v e n i e n t f o r m for t h e c u r r e n t a p p l i c a t i o n .. 13.

(14) •. C a l c u l a t i o n s a n d g e n e r a t i o n , of t h e c o n t r o l o r d e r s t o t h e m a n i p u l a t o r s , a c c o r d i n g t o p h a s e n u m b e r 2.. F i g u r e 2 shows t h e c o m p o n e n t s of software for a r t i f i c i a l v i s i o n s y s t e m .. F i g u r e 2: C o m p o n e n t s of software for a r t i f i c i a l v i s i o n s y s t e m s A c c o r d i n g t o [42], t h e s t e r e o v i s i o n b a s e d c o n f i r m a t i o n consists i n f o u r m a j o r steps: •. d e t e r m i n a t i o n of regions of i n t e r e s t i n t h e stereoscopic. images.. •. a p p l i c a c i o n of a n u m e r i c a l z o o m t o m a x i m i z e t h e d e t e c t i o n r a n g e .. •. c o m p u t a t i o n of a l o c a l d i s p a r i t y m a p i n t h e regions of i n t e r e s t .. •. c r i t e r i o n e v a l u a t i o n f r o m t h i s d i s p a r i t y m a p t o c o n f i r m t h e e x i s t e n c e of a n obsta¬ cle.. A u t o n o m o u s m o b i l e r o b o t s m a k e use of stereo v i s i o n t o m e a s u r e t h e i r r e l a t i v e d i s t a n c e to obstacles.. T h i s m e t h o d is also r e l a t i v e l y i n e x p e n s i v e ; w h i l e laser scanners c a n cost. tens of t h o u s a n d s of d o l l a r s , stereo v i s i o n r e q u i r e s o n l y t w o a l i g n e d c a m e r a s a n d some processing power.. S t e r e o v i s i o n is a t e c h n i q u e t h a t uses t w o c a m e r a s t o m e a s u r e dis¬. t a n c e s f r o m t h e c a m e r a s , s i m i l a r t o h u m a n d e p t h p e r c e p t i o n w i t h h u m a n eyes. p r o c e s s uses t w o p a r a l l e l c a m e r a s a l i g n e d at a k n o w n d i s t a n c e of s e p a r a t i o n . c a m e r a c a p t u r e s a n i m a g e a n d these i m a g e s are a n a l y z e d for c o m m o n features. The Each [12].. T r i a n g u l a t i o n is u s e d w i t h t h e r e l a t i v e p o s i t i o n of these m a t c h e d p i x e l s i n t h e i m a g e s as seen i n F i g u r e 3.. 14.

(15) F i g u r e 3: S t e r e o V i s i o n T r i a n g u l a t i o n T r i a n g u l a t i o n r e q u i r e s k n o w i n g t h e f o c a l l e n g t h of t h e c a m e r a (f),. the distance. b e t w e e n t h e c a m e r a bases (b), a n d t h e center of t h e i m a g e s o n t h e i m a g e p l a n e (c. 1. c ).. and. D i s p a r i t y (d) is t h e difference b e t w e e n t h e l a t e r a l d i s t a n c e s t o t h e f e a t u r e p i x e l (v. 2. 2. a n d v i ) o n t h e i m a g e p l a n e f r o m t h e i r r e s p e c t i v e centers. U s i n g t h e c o n c e p t of s i m i l a r t r i a n g l e s , t h e d i s t a n c e f r o m t h e c a m e r a s (D). is c a l c u l a t e d as. T h e r e s u l t of t h e c o m p u t e r v i s i o n s y s t e m is a d e p t h field m a p w h i c h is a g r a y s c a l e i m a g e of t h e same size t o t h e o r i g i n a l i m a g e . E a c h g r a y s c a l e i m a g e represents a d i s t a n c e f r o m t h e c a m e r a s . F o r e x a m p l e , a w h i t e p i x e l signifies a p i x e l i n t h e c o m p u t e r ' s v i s i o n near infinity while a black p i x e l means a point i n the infinity distance.. 2.2. Road Detection and Image Processing. V i s i o n - b a s e d r o a d d e t e c t i o n is a n i m p o r t a n t r e s e a r c h t o p i c i n different areas of com¬ p u t e r v i s i o n s u c h as a u t o n o m o u s d r i v i n g , c a r c o l l i s i o n w a r n i n g a n d p e d e s t r i a n c r o s s i n g detection.. [3]. S o m e b a s i c c o n c e p t s u s e d i n i m a g e p r o c e s s i n g are: •. T h r e s h o l d : is a m e t h o d of i m a g e s e g m e n t a t i o n . F r o m a g r a y s c a l e i m a g e , thresh¬ o l d i n g c a n be u s e d t o create b i n a r y i m a g e s . D u r i n g t h e p r o c e s s , i n d i v i d u a l p i x e l s i n a n i m a g e are m a r k e d as " o b j e c t p i x e l s " i f t h e i r v a l u e is g r e a t e r t h a n some t h r e s h o l d v a l u e a n d as a " b a c k g r o u n d " p i x e l o t h e r w i s e .. •. C o n t r a s t : is t h e difference i n v i s u a l p r o p e r t i e s t h a t m a k e s a n o b j e c t d i s t i n g u i s h ¬ able f r o m o t h e r o b j e c t s a n d t h e b a c k g r o u n d .. 15.

(16) •. B r i g h t n e s s : is a n a t t r i b u t e of v i s u a l p e r c e p t i o n i n w h i c h a source a p p e a r s t o be r a d i a t i n g or r e f l e c t i n g l i g h t . I n o t h e r w o r d s , b r i g h t n e s s is t h e p e r c e p t i o n e l i c i t e d b y t h e l u m i n a n c e of a v i s u a l t a r g e t . I n t h e R G B c o l o r space, b r i g h t n e s s c a n be t h o u g h t of as t h e a r i t h m e t i c m e a n [i of t h e r e d , green a n d b l u e c o l o r c o o r d i n a t e s :. 2.3. Fuzzy Control. T h e c o n t r o l l e r s are t h e b r a i n of t h e r o b o t , t h e y do a l l t h e n e c e s a r y c a l c u l a t i o n s so t h a t t h e r o b o t does w h a t is d e s i r e d . I n t h i s case, t h e c o n t r o l l e r is r e q u i r e d t o m a i n t a i n t h e m o b i l e at a c e r t a i n s p e e d a n d p o s i t i o n . F u z z y l o g i c was first d e s i g n e d t o represent a k n o w l e d g e e x p r e s s e d i n a l i n g u i s t i c o r v e r b a l f o r m [21]. 2.3.1. D e f i n i t i o n of the F u z z y Sets. A w a y t o define a set is t o e n u m e r a t e t h e i r e l e m e n t s , a n d t h e o t h e r , t o use a f u n c t i o n P(x),. w h e r e e v e r y e l e m e n t x of t h e set has a p r o p e r t y P.. A t h i r d w a y t h a t is t o define a. set A u s i n g a c h a r a c t e r i s t i c f u n c t i o n . L e t A define t h e d o m a i n X.. T h e n \i. A. [0,1]. : X —. is a c h a r a c t e r i s t i c f u n c t i o n of t h e set A i f e v e r y x. T h e g e n e r a l i z a t i o n of t h e t h e o r y of f u z z y sets is: t h e m e m b e r s h i p f u n c t i o n /i. p. f u z z y set F is a f u n c t i o n [i. p. 2.3.2. : U —. of a. [0,1]. O p e r a t i o n s w i t h F u z z y Sets. N o t i o n s l i k e e q u a l i t y a n d i n c l u s i o n are t w o f u z z y sets d e r i v e d f r o m t h e classic t h e o r y of sets. T w o f u z z y sets are e q u a l i f e v e r y e l e m e n t i n t h e u n i v e r s e has t h e same m e m b e r s h i p degree i n e v e r y one of t h e m . A f u z z y set A is a subset of t h e set B i f e v e r y e l e m e n t i n t h e u n i v e r s e has a s m a l l e r degree of m e m b e r s h i p . I n t h e classic t h e o r y , t h e u n i o n , i n t e r s e c t i o n a n d c o m p l e m e n t. are sets of s i m p l e. o p e r a t i o n s t h a t are c l e a r l y defined. E v e r y l o g i c o p e r a t o r a n d , or a n d n o t has a s e m a n t i c w e l l d e f i n e d a c c o r d i n g t o t h e p r o p o s i t i o n a l l o g i c . T h e m o s t u s e d o p e r a t o r s i n f u z z y sets. 16.

(17) T h i s is a s i m p l e e x t e n s i o n of t h e c l a s s i c o p e r a t i o n s . O t h e r p o s s i b l e e x t e n s i o n s are, iAns(x)=. 1A(X). • is(x). o r J A U B ( x ) = min((1,jA(x). + 1B(x)). .. M o r e g e n e r a l l y , t r i a n g u l a r n o r m s ( T - n o r m a n d S - n o r m ) are u s e d t o represent inter¬ section, u n i o n or complement. 2.3.3. Fuzzy Relations. A r e l a t i o n c a n be c o n s i d e r e d as a set of o r d e r e d p a i r s . A s classic sets, c l a s s i c r e l a t i o n s c a n be d e s c r i b e d b y a m e m b e r s h i p f u n c t i o n . defined i n X. ...X. 1. A for e v e r y X. n. 1. 2.3.4. then J. R. = X. 1. ...X. n. S u p p o s e t h a t R is a r e l a t i o n of n o r d e r. — [0,1]. is a m e m b e r s h i p f u n c t i o n of t h e set. ...X , n. Operations with Fuzzy Relations. T h e t w o m o s t u s e d o p e r a t i o n s i n f u z z y r e l a t i o n s are i n t e r s e c t i o n a n d u n i o n . T h e s e are d e f i n e d as f o l l o w : let R a n d S be b i n a r y r e l a t i o n s defines i n X. x Y.. T h e intersection. b e t w e e n R a n d S is d e f i n e d as. I n s t e d of m i n i m u m , a n y T - n o r m c a n b e u s e d . T h e u n i o n of R a n d S is d e f i n e d as. I n s t e a d of m a x i m u m , a n y S - n o r m c a n b e u s e d . T h e s e d e f i n i t i o n s c a n b e. extended. t o a n y n u m b e r of r e l a t i o n s . T h e c o m b i n a t i o n of f u z z y sets a n d f u z z y r e l a t i o n s is c a l l e d c o m p o s i t i o n a n d is defines as: let A be a f u z z y set d e f i n e d i n X. a n d R a f u z z y r e l a t i o n d e f i n e d i n XxY.. Then,. t h e c o m p o s i t i o n A a n d R r e s u l t s i n a f u z z y set B d e f i n e d i n Y is g i v e n b y. T h i s is t h e so c a l l e d m a x - m i n c o m p o s i t i o n . T h e m a x - p r o d c o m p o s i t i o n is d e f i n e d as. 2.3.5. Approximate Reasoning. T h e a p p r o x i m a t e r e a s o n i n g is t h e best way, i n w h i c h , t h e f u z z y l o g i c covers a v a r i e t y of i n f e r e n c e r u l e s w h e r e t h e p r e m i s e s c o n t a i n f u z z y p r o p o s i t i o n s .. 17.

(18) Inference i n a p p r o x i m a t e r e a s o n i n g is i n c o n t r a s t t o t h e inference i n classic l o g i c . I n a p p o x i m a t e r e a s o n i n g , t h e consecuence of a set g i v e n t h e f u z z y p r o p o s i t i o n s d e p e n d s o n a e s e n c i a l w a y i n t h e aggregate m e a n i n g t o these f u z z y p r o p o s i t i o n s .. T h e n , inference. i n a p p o x i m a t e r e a s o n i n g is t h e p r o c e s s i n g of f u z z y sets t h a t represent t h e m e a n i n g of a c e r t a i n set of f u z z y p r o p o s i t i o n s . and J. B. F o r e x a m p l e , g i v e n t h e m e m b e r s h i p f u n c t i o n s JA. , r e p r e s e n t i n g t h e m e a n i n g of a f u z z y p r o p o s i t i o n X is A a n d t h e m e a n i n g of t h e. f u z z y c o n d i t i o n a l If. X is A Then. Y is B, t h e m e m b e r s h i p f u n c t i o n c a n be. r e p r e s e n t i n g t h e m e a n i n g of t h e c o n c l u s i o n Y is 2.3.6. Inference. computed. B.. Rules. I n a p p r o x i m a t e r e a s o n i n g , t w o inference rules are t h e m o s t i m p o r t a n t . T h e inference c o m p o s i t i o n r u l e a n d t h e g e n e r a l i z e d m o d u s p o n e n s . T h e first r u l e uses a f u z z y r e l a t i o n t o e x p l i c i t i l y represent t h e c o n e c t i o n b e t w e e n t w o f u z z y p r o p o s i t i o n s , t h e s e c o n d a r u l e I F - T H E N t h a t i m p l i c i t l y represents a f u z z y r e l a t i o n .. T h e generalized. uses. modus. p o n e n s has t h e inference s y m b o l i c scheme. where S i a n d S. 2. are s y m b o l i c n a m e s for l i n g u i s t i c v a r i a b l e s , a n d P ,. P,. 1. 2. Q. 1. and. Q. 2. are s y m b o l i c n a m e s for l i n g u i s t i c v a l u e s . T h e r u l e of c o m p o s i t i o n for inference c a n be c o n s i d e r e d as a s p e c i a l case of g e n e r a l i z e d m o d u s p o n e n s .. T h e general symbolic f o r m. is. w h e r e S RS 1. 2. is r e a d as " S. 1. s i n i n r e l a t i o n R t o S " a n d t h e m e a n i n g is r e p r e s e n t e d 2. as a f u z z y r e l a t i o n . T h e n , i n s t e a d of t h e I F - T H E N r u l e , t h e r e is a f u z z y r e l a t i o n R. N o w t h e scheme of inference is c o n s i d e r e d ,. w h e r e P a n d Q are f u z z y sets r e p r e s e n t i n g t h e m e a n i n g of P, Q a n d R are a f u z z y r e l a t i o n d e f i n i n g t h e m e a n i n g of R , P is d e f i n e d i n X. a n d R over X. x. Y.. T h e n the. c a l c u l a t i o n of t h e c o m p o s i t i o n a l r u l e of inference is d o n e as. 2.3.7. The I F - T H E N. Rules. T h e r e are a n u m b e r of r e l a t i o n s t h a t c a n represent t h e m e a n i n g of I F X is A T H E N Y is B.. T h e m o s t u s e d i m p l i c a t i o n s i n f u z z y sets are:. I m p l i c a t i o n of L u k a s i e w i c z : T h i s i m p l i c a t i o n is b a s e d o n t h e e q u a l i v a l e n c e p — q =' p V q. T o represent O R is also p o s s i b l e t o use t h e l i m i t e d s u m min(l, of t h e m a x i m u m max(p,. 1 — p + q) i n s t e a d. q). T h i s r e s u l t s i n t h e r e l a t i o n c a l l e d R , d e f i n e d as a. 18.

(19) I m p l i c a t i o n of Z a d e h :. I n l o g i c of t w o v a l u e s , p — q has t h e same t r u e values as. (pAq) V ' p . T h i s e q u i v a l e n c e was u s e d b y Z a d e h i n t h e n e x t f o r m. I m p l i c a t i o n of M a m d a n i : W i t h respect t o f u z z y c o n t r o l , t h i s is t h e m o s t i m p o r t a n t i m p l i c a t i o n k n o w n i n t h e l i t e r a t u r e . Its d e f i n i t i o n is b a s e d i n t h e i n t e r s e c t i o n . relation R. c. 2.3.8. The. (x of c o n j u c t i o n ) is d e f i n e d as. Fuzzification ( F M ). A c c o r d i n g t o [24], t h e f u z z y f i c a t i o n m o d u l e does t h e n e x t f u n c t i o n s : •. F M - F 1 : D o e s a scale t r a n s f o r m a t i o n ( i n p u t n o r m a l i z a t i o n ) t h a t m a p s t h e p h y s i c a l values of t h e i n p u t v a r i a b l e s i n a n o r m a l i z e d u n i v e r s e ( n o r m a l i z e d d o m a i n ) .. It. also m a p s t h e n o r m a l i z e d v a l u e of t h e o u t p u t v a r i a b l e i n t h e p h y s i c a l d o m a i n . W h e n a n o r m a l i z e d d o m a i n is u s e d , i t is n o n e c e s s a r y t o use F M - F 1 . •. F M - F 2 : D o e s t h e d e n a z i f i c a t i o n t h a t c o n v e r t s t h e recent v a l u e of a n i n p u t v a r i a b l e of a f u z z y set t o m a k e i t c o m p a t i b l e w i t h t h e r e p r e s e n t a t i o n of t h e f u z z y set of the i n p u t variable.. 2.3.9. Inference. Machine. T h e r e are t w o w a y s t o a t t a c k t h e d e s i g n of t h e inference m a c h i n e of a f u z z y c o n t r o l l e r : (1) inference b a s e d i n c o m p o s i t i o n a n d (2) inference b a s e d o n i n d i v i d u a l r u l e s .. The. b a s i c f o r m of t h e inference m a c h i n e of t h e s e c o n d t y p e is t o c o m p u t e t h e g e n e r a l v a l u e of t h e o u t p u t v a r i a b l e i n i n d i v i d u a l c o n t r i b u t i o n s for e a c h r u l e i n t h e base of r u l e s . E a c h i n d i v i d u a l c o n t r i b u t i o n represents t h e values of t h e c o m p u t e d o u t p u t v a r i a b l e s for e a c h i n d i v i d u a l r u l e . T h e o u t p u t of t h e f u z z i f i c a t i o n m o d u l e , represents t h e a c t u a l values of t h e i n p u t v a r i a b l e s a n d are p r o j e c t e d t o e a c h r u l e , a n d a c e r t a i n degree of e q u i v a l e n c e is s t a b l i s h e d . E a c h degree of e q u i v a l e n c e represents t h e degree of s a t i s f a c t i o n of a f u z z y p r o p o s i t i o n . B a s e d o n t h e degree of e q u i v a l e n c e , t h e v a l u e of t h e o u t p u t v a r i a b l e i n t h e l a s t r u l e is m o d i f i e d . T h e set of a l l t h e o u t p u t values of t h e e q u i v a l e n t rules represents t h e g e n e r a l v a l u e of t h e f u z z y o u t p u t . I n t h i s c o n t e x t , t h e d e s i g n of p a r a m e t e r s for t h e inference m a c h i n e is: •. C h o o s e t h e r e p r e s e n t a t i o n of t h e m e a n i n g for a single r u l e ,. •. C h o o s e t h e r e p r e s e n t a t i o n of t h e m e a n i n g for a set of r u l e s ,. •. C h o o s e t h e inference m a c h i n e ,. •. P r o v e t h e set of rules t o be c o n s i s t e n t a n d c o m p l e t e . 19.

(20) 2.3.10. Defuzzification. (DM). A c c o r d i n g t o [24], t h e f u n c t i o n s of t h e d e f u z z i f i c a t i o n m o d u l e are: •. D M - F 1 : D o e s t h e so c a l l e d d e f u z z i f i c a t i o n t h a t c o n v e r t s t h e set of m o d i f i e d o u t p u t values t o a single v a l u e .. •. D M - F 2 : D o e s t h e d e n o r m a l i z a t i o n of t h e o u t p u t t h a t m a p s t h e o u t p u t p o i n t s i n t h e p h y s i c a l d o m a i n . D M - F 2 is n o t n e c e s s a r y i f d o m a i n s n o t n o r m a l i z e d are u s e d .. T w o of t h e m o s t u s e d d e f u z z i f i c a t i o n o p e r a t o r s are C e n t e r of A r e a a n d M e a n of M a x i ¬ mum. C e n t e r of A r e a : T h e m e t h o d of center of a r e a or g r a v i t y center is t h e m o s t k n o w n d e f u z z i f i c a t i o n m e t h o d . It is a l m o s t d i s c r e t e , t h i s r e s u l t s i n. I n t h e c o n t i n u e s case, i t is o b t a i n e d. M e a n of M a x i m u m : T h i s m e t h o d d e t e r m i n e s t h e first a n d last values w h e r e Y has a m a x i m u m degree of m e m b e r s h i p a n d t h e n i t t a k e s t h e m e a n of those t w o v a l u e s . Formally,. 2.4. Real-Time Systems. D e a l i n g w i t h c o m p u t e r i m a g e p r o c e s s i n g is a c h a l l e n g i n g t a s k i n v e h i c l e n a v i g a t i o n since h i g h n u m b e r of c o m p u t a t i o n s n e e d t o be d o n e i n o r d e r t o p r o c e s s t h e i n f o r m a t i o n f r o m t h e c a m e r a s a n d s t i l l have t i m e t o c o n t r o l t h e v e h i c l e a r o u n d t h e d e s i r e d t r a j e c t o r y . R e a l - t i m e c o m p u t i n g s y s t e m s are s y s t e m s i n w h i c h t h e i m p o r t a n c e of a n a c t i o n is n o t o n l y t h a t i t is d o n e c o r r e c t , b u t also t h e t i m e i t t a k e s t o be p r o c e s s e d .. I n o r d e r for. t a s k s t o get d o n e at e x a c t l y t h e r i g h t t i m e , r e a l - t i m e s y s t e m s m u s t a l l o w y o u t o p r e d i c t a n d c o n t r o l w h e n t a s k s o c c u r [1]. A r e a l - t i m e s y s t e m m u s t d e m o n s t r a t e t h e f o l l o w i n g features: •. P r e d i c t a b l y fast response t o u r g e n t events.. •. H i g h degree of s c h e d u l a b i l i t y : t h e t i m i n g r e q u i r e m e n t s of t h e s y s t e m m u s t. be. s a t i s f i e d at h i g h degrees of resource usage. •. S t a b i l i t y u n d e r t r a n s i e n t o v e r l o a d : w h e n t h e s y s t e m is o v e r l o a d e d b y events a n d i t is i m p o s s i b l e t o meet a l l t h e d e a d l i n e s , t h e d e a d l i n e s of selected c r i t i c a l t a s k s must still be guaranteed. 20.

(21) 2.4.1. R e a l - T i m e Scheduling Policies. T h e r e are different schemes for s c h e d u l i n g events. A c c o r d i n g t o [1], some p o p u l a r r e a l t i m e scheduling policies include: •. F i x e d P r i o r i t y P r e e m p i t v e S c h e d u l i n g : E v e r y t a s k has a fixed p r i o r i t y t h a t does n o t c h a n g e unless t h e a p p l i c a t i o n s p e c i f i c a l l y changes i t . A h i g h e r - p r i o r i t y t a s k preempts a lower-priority task.. M o s t real-time o p e r a t i n g systems support this. scheme. •. D y n a m i c - P r i o r i t y P r e e m p t i v e S c h e d u l i n g : T h e p r i o r i t y of a t a s k c a n c h a n g e f r o m i n s t a n c e t o i n s t a n c e or w i t h i n t h e e x e c u t i o n of a n i n s t a n c e , i n o r d e r t o meet a specific r e s p o n s e t i m e o b j e c t i v e . A h i g h e r - p r i o r i t y t a k s p r e e m p t s a l o w e r - p r i o r i t y t a s k . V e r y few c o m m e r c i a l r e a l - t i m e o p e r a t i n g s y s t e m s s u p p o r t s u c h p o l i c i e s .. •. Rate-Monotonic Scheduling: A n optimal. fixed-priority. p r e e m p t i v e s c h e d u l i n g pol¬. i c y i n w h i c h , t h e h i g h e r t h e f r e q u e n c y of a p e r i o d i c t a s k , t h e h i g h e r i t s p r i o r i t y . T h i s p o l i c y assumes t h a t t h e d e a d l i n e of a p e r i o d i c t a s k is t h e s a m e as i t s pe¬ riod.. It c a n be i m p l e m e n t e d i n a n y o p e r a t i n g s y s t e m s u p p o r t i n g. fixed-priority. p r e e m p t i v e s c h e d u l i n g or g e n e r a l i z e d t o a p e r i o d i c t a s k s . •. D e a d l i n e - M o n o t o n i c S c h e d u l i n g : A g e n e r a l i z a t i o n of t h e r a t e - m o n o t o n i c. schedul-. i n g p o l i c y i n w h i c h t h e d e a d l i n e of a t a s k is a fixed p o i n t i n t i m e r e l a t i v e t o b e g i n n i n g of a p e r i o d . ity.. T h e s h o r t e r t h i s (fixed) d e a d l i n e , t h e h i g h e r i t s prior¬. W h e n t h e d e a d l i n e t i m e equals t h e p e r i o d , t h i s p o l i c y is i d e n t i c a l t o t h e. rate-monotonic scheduling policy. •. Earliest-Deadline-First Scheduling: policy.. A dynamic-priority preemptive. scheduling. T h e d e a d l i n e of a t a s k i n s t a n c e is t h e a b s o l u t e p o i n t i n t i m e b y w h i c h. t h e i n s t a n c e m u s t c o m p l e t e . T h e d e a d l i n e is c o m p u t e d w h e n t h e i n s t a n c e m u s t c o m p l e t e . T h e s c h e d u l e r p i c k s t h e t a s k w i t h t h e earliest d e a d l i n e t o r u n. first.. A. task w i t h an earlier deadline preempts a task w i t h a later deadline. T h i s policy m i n i m i z e s t h e m a x i m u m lateness of anyset of t a s k s r e l a t i v e t o a l l o t h e r s c h e d u l i n g policies. •. Least Slack Scheduling:. A dynamic-priority non-preemptive. scheduling policy.. T h e s l a c k of a t a s k i n s t a n c e is i t s a b s o l u t e d e a d l i n e m i n u s t h e r e m a i n i n g w o r s t case e x e c u t i o n t i m e for t h e t a s k i n s t a n c e t o c o m p l e t e . t a s k w i t h t h e shortest s l a c k t o r u n. first.. lateness of a n y set of t a s k s .. 21. T h e scheduler picks the. T h i s policy maximizes the m i n i m u m.

(22) 3. State of A r t. I n t h i s c h a p t e r , r e l a t e d w o r k s are p r e s e n t e d i n o r d e r t o s t a b l i s h t h e c o n t e x t o n w h i c h t h i s thesis was b a s e d . T h e m a i n areas of s t u d y of t h i s thesis are: r o a d d e t e c t i o n u s i n g stereo v i s i o n a n d f u z z y c o n t r o l i n a u t o n o m o u s m o b i l e vehicles.. 3.1. Road Detection with Stereo Vision. V i s i o n - b a s e d t r a j e c t o r y d e t e c t i o n is a v e r y i m p o r t a n t a r e a of s t u d y b e c a u s e i t is a f u n d a m e n t a l part i n autonomous d r i v i n g , car collision w a r n i n g , object detection a n d pedestrian crossing detection. D e t e c t i n g trajectories a n d roads w i t h vision systems can b e d o n e u s i n g m o n o c u l a r v i s i o n - s y s t e m s a n d stereo v i s i o n s y s t e m s [15, 3]. P a r t i c u l a r l y , stereo v i s i o n has b e e n s t u d i e d l a t e l y b e c a u s e i t gives a n a d v a n t a g e over m o n o c u l a r v i s i o n s y s t e m s : t h e m e a s u r e m e n t of d i s t a n c e , w i t h o u t t h e n e e d of m o r e sensors; therefore, we focus o n t h e r e s e a r c h of stereo v i s i o n t o d e t e c t t h e t r a j e c t o r y w h e r e t h e v e h i c l e is d r i v e n . G r e a t i n t e r e s t has r e c e n t l y a r i s e n i n t h e d e s i g n a n d d e v e l o p m e n t of a u t o n o m o u s v e h i c l e [34, 35].. T w o of f u n c t i o n s of A L V ( A u t o n o m o u s L a n d V e h i c l e ). land. autonomous. n a v i g a t i o n are t h e o b s t a c l e d e t e c t i o n a n d t h e r o b u s t d e t e c t i o n a n d t r a c k i n g of r o a d boundaries. R o a d d e t e c t i o n is a c r u c i a l p r o b l e m for i n t e l l i g e n t vehicles a n d m o b i l e r o b o t s .. It. p r o v i d e s i n f o r m a t i o n a b o u t t h e w o r l d t h a t enables t h e i n t e l l i g e n t v e h i c l e or r o b o t t o i n t e r a c t w i t h i t s e n v i r o m e n t a n d react t o events or changes t h a t i n f l u e n c e i t s t a s k [44]. M a n y researchers h a v e b e e n s t u d y i n g i t for s e v e r a l decades a n d d r a m a t i c d e v e l o p m e n t has b e e n a c c o m p l i s h e d , w h i c h c a n b e c a t e g o r i z e d i n t o t w o m a i n t y p e s of. methods:. vision-based methods and L I D A R ( L i g h t Detection A n d Ranging)-based methods.. The. s t e r e o v i s i o n m a k e s p o s s i b l e t o use o n l y c a m e r a s t o d i r e c t l y m e a s u r e r a n g e a n d c o l o r information, just like h u m a n operators.. T h e r e f o r e , v i s i o n - b a s e d r o a d d e t e c t i o n is a. v e r y i m p o r t a n t as w e l l as p r o m i s i n g b r a n c h i n t h e field [15]. A m o n g t h e c u r r e n v i s i o n b a s e d m e t h o d s , some use m o n o c u l a r c a m e r a t o e x t r a c t t h e r o a d r e g i o n b y e m p l o y i n g features w i t h specific i n t e n s i t y , c o l o r a n d t e x t u r e w h i l e o t h e r s use a b i n o c u l a r c a m e r a for r o a d d e t e c t i o n b y u s i n g 3 D s t r u c t u r a l i n f o r m a t i o n [15]. A v a r i e t y of m e t h o d s have b e e n p r o p o s e d for o b s t a c l e d e t e c t i o n .. Several kinds. of sensors are u s e d t o a c q u i r e i n f o r m a t i o n f r o m t h e e n v i r o n m e n t t o c a r r y o u t r o b o t n a v i g a t i o n w i t h r e a l - t i m e o b s t a c l e a v o i d a n c e . V i s i o n s y s t e m , 2 D or 3 D laser r a n g e f i n d e r a n d c o m b i n a t i o n s of t h e m are u s e d t o d e t e c t o b s t a c l e s u n d e r different e n v i r o n m e n t . S t e r e o v i s i o n t e c h n i q u e [43] was p o p u l a r l y u s e d t o d e t e c t o b s t a c l e s for A L V . T h e m a i n p r o b l e m of stereo v i s i o n is t h a t c o m p l e x a l g o r i t h m has t o be u s e d t o g u a r a n t e e t h e c o r r e c t p i x e l m a t c h i n g b e t w e e n t w o i m a g e s . I n t h e p a s t s e v e r a l y e a r s , t h e laser r a n g e m e a s u r e m e n t s y s t e m has b e e n u s e d t o d e t e c t o b s t a c l e .. I n [58], a n o b s t a c l e. detection. s y s t e m for A L V u n d e r s e m i - s t r u c t u r a l e n v i r o n m e n t w i t h t w o 2 D laser r a n g e finders is d e s c r i b e d . 3 D L R F ( L a s e r R a n g e F i n d e r ) [ 3 8 ] a n d q u a s i - 3 D L R F [26] is u s e d for o b s t a c l e detection i n cross-country environment and u r b a n environment. A u t o n o m o u s r o a d f o l l o w i n g c a n be c o n s i d e r e d as five different p a r t s : h o r i z o n esti¬ m a t i o n , o b s t a c l e d e t e c t i o n , o b s t a c l e a v o i d a n c e , r o a d d e t e c t i o n a n d 3 D v i e w as s h o w n. 22.

(23) unpredictable situations and imaging conditions such as extreme shadows, illumination, and complex road shapes (Fig. 1). The proposed algorithm exploits 3D information available in a single image to detect the drivable road surin figure face 4.ahead of the target vehicle (Fig. 2). In this way, we define different 3D contextual cues such as horizon line (road should be below of it), vanishing point (where roads are aimed at), 3D layout (side walks, buildings and sky), and 3D stages (road models). The novelty of the approach is the introduction of 3D contextual cues and combining them to obtain a diversified ensemble Figure of road cues. In general, combining multiple clas4: Components of autonomous road following sifiers is a powerful technique to improve the performance 3.1.1 Horizon Estimation[9, 7]. The improvement is even higher of single classifiers Thewhen horizonthe line method is important information for knowing thei.e., area of interest in the are image. uses diversified cues, cues which The road will be usually below the horizon line. To estimate the position of the horizon robust or sensitive different artifacts in anestimates image.the line, an approach has been to introduced by [32, 28, 52].present This method horizon line by applying mixtures of linear regressorsinformation to the description In this way, the non-linear proposed method extracts at of an image obtained using gist descriptors [4]. Also, the horizon line can be easily detected scene, image and pixel–level. Further, the proposed method by detecting the road lines and extend them to know the point where those two lines exploits the sequential of the data thethat intersect; above that point, therenature is no information aboutby theconsidering road, and below point is the area of interest. A vanishing point is computed at that intersection to existing correlation between detected roads in consecutive differentiate the road from the horizon gradually [3]. Figure 5 shows the horizon line frames. estimation according to [3].. Furthe and mi. 2.1. H. The where low the line, th method mixtur obtaine is com A fuzz the hor horizon. Figure put ima bility m exhibiti Detection algorithm of the horizonexploits line according to [3]. Figure 2. Figure The 5: proposed all the information available in a single image to detect the drivable road surface ahead Detecting the road using the vanishing point is robust to global lightning variations, the target vehicle. different road types, damaged roads and the presence of other vehicles in the scene. However, it is not robust against curved roads, heavy traffic and when strong shadow edges are present [3].. The rest of the paper is organized as follows. First, in 23. 2.2. V. The points exploit.

(24) 3.1.2. Obstacle. Detection. W h e n d e a l i n g w i t h o b s t a c l e s , t h e r e are a n u m b e r of w a y s t o d e t e c t t h e m . O n e m e t h o d is t o use r o a d - b a r r i e r s [37], a n o t h e r m e t h o d is t o use u n c e r t a i n t y [53] a n d t h e p r i n c i p l e of c o l o r d e c l i v i t y [11]. A c c o r d i n g t o [24], t h e obstacles c a n be classified as p o s i t i v e a n d n e g a t i v e . P o s i t i v e obstacles are tree t r u n k s , s a n d d u n e s a n d o t h e r s t h a t e x t e n d o u t of t h e g r o u n d surface. T h e o t h e r t y p e are t h e n e g a t i v e o b s t a c l e s , s u c h as d i t c h e s or holes, t h a t e x t e n d i n t o t h e g r o u n d p l a n e . P o s i t i v e obstacles are d e t e c t e d b y a p p l y i n g a hysteresis t h r e s h o l d o n t h e m e a s u r e d t e r r a i n slope a r o u n d i m a g e p o i n t s a c c o r d i n g t o [24]. N e g a t i v e obstacles are d e t e c t e d b y l o o k i n g for d e p t h j u m p s i n t h e r a n g e profile of a n i m a g e c o l u m n .. In. o r d e r t o f o l l o w r o a d s i d e u n d e r v a r i o u s c o n d i t i o n , v i s i o n - b a s e d sensor a n d r a n g e - b a s e d sensors are u s e d . I n [44], a n o b s t a c l e d e t e c t i o n m e t h o d i n t e g r a t e s i n f o r m a t i o n f r o m laser rangefinder a n d c a m e r a to detect a n d track obstacles.. In[3], a r o a d f o l l o w i n g m e t h o d. i n t e g r a t e s i n f o r m a t i o n f r o m laser r a n g e f i n d e r a n d c a m e r a t o d e t e c t a n d t r a c k t h e r o a d b o u n d a r y . R o a d h e i g h t , s m o o t h n e s s , c o l o r , a n d t e x t u r e were c o m b i n e d t o y i e l d h i g h e r p e r f o r m a n c e of r o a d s i d e . 3.1.3. 3D. View. A n o t h e r 3 D cue is t h e l a y o u t of t h e scene. T h e l a y o u t is a n a l y z e d u s i n g t h r e e m a j o r p a r t s of t h e i m a g e : (1) s k y p i x e l s , (2) v e r t i c a l surface p i x e l s a n d (3) g r o u n d p i x e l s . W i t h these 3 D cues t h e r o a d is l i m i t e d t o g r o u n d , n o n - s k y i m a g e r e g i o n s .. F u r t h e r , regions. are a v o i d e d w h i c h are v e r t i c a l l y o r i e n t a t e d (i.e., b u i l d i n g s , v e h i c l e s , p e d e s t r i a n s o r a n y o t h e r o b j e c t present i n t h e scene). T h e s e g m e n t a t i o n of t h e i m a g e i n these 3 D cues is c o m p u t e d b y t h e m e t h o d p r o p o s e d i n [16]. R o a d d e t e c t i o n u s i n g scene l a y o u t is r o b u s t t o different t y p e s of a s p h a l t s , l a n e m a r k i n g s a n d p e d e s t r i a n crossings. H o w e v e r , scenel a y o u t for r o a d d e t e c t i o n m a y b e s e n s i t i v e t o s h a d o w s as t h e a l g o r i t h m uses s u p e r p i x e l s e g m e n t a t i o n [3].. A n o t h e r i m p o r t a n t cue for d e t e c t i n g t h e r o a d is i t s 3 D. geometry.. T h i s r o a d g e o m e t r y c a n be i n f e r r e d u s i n g a scene (road) c l a s s i f i c a t i o n a l g o r i t h m w h e r e e a c h class represents t y p i c a l 3 D r o a d geometries s u c h as left t u r n , s t r a i g h t r o a d a n d j u n c t i o n s [30].. 3.2. Fuzzy Control i n Autonomous Mobile Vehicles. C o n t r o l a l g o r i t h m s s h o u l d be c o n s i d e r e d as a n i m p o r t a n t issue i n r o a d f o l l o w i n g t o ensure safe a n d s m o o t h r i d e s .. A l t h o u g h a l o t of researches have b e e n d o n e , m o s t of. t h e m are b a s e d o n t r a d i t i o n a l c o n t r o l t h e o r y s u c h as P I D [56] a n d l i n e a r c o n t r o l l e r s [6]. T h e k i n e m a t i c b e h a v i o r of a u t o n o m o u s r o a d f o l l o w i n g is t y p i c a l l y n o n l i n e a r . T h e r e f o r e l i n e a r m o d e l s u s u a l l y f a i l t o d e s c r i b e these s y s t e m s efficiently.. H o w e v e r , i t is diffi¬. c u l t t o a n a l y z e n o n l i n e a r m a t h e m a t i c a l m o d e l s for a u t o n o m o u s r o a d f o l l o w i n g schemes. O t h e r m e t h o d s s u c h as n e u r a l - n e t w o r k s [5, 17] a n d r e i n f o r c e m e n t l e a r n i n g ( R L ). [48]. a p p r o a c h e s have also b e e n u s e d i n r o a d f o l l o w i n g , b u t these a p p r o a c h e s r e q u i r e learn¬ i n g w h i c h consumes e x t r a c o m p u t a t i o n t i m e . H u m a n drivers can drive a car s m o o t h l y w i t h their d r i v i n g expertise rather t h a n knowledge about control theory. 24. F u z z y logic.

(25) c o n t r o l is k n o w n t o b e a n o r g a n i z e d m e t h o d t o e m u l a t e h u m a n e x p e r t i s e i n d e a l i n g w i t h i m p r e c i s e d a t a . It a t t e m p t s t o a p p l y a h u m a n - l i k e w a y of t h i n k i n g i n t h e appli¬ c a t i o n areas a n d a l l o w s l i n g u i s t i c t e r m s for i n t e r m e d i a t e values t o b e d e f i n e d besides c o n v e n t i o n a l e v a l u a t i o n s . F u z z y l o g i c c o n t r o l has been[10] a p p l i e d i n a u t o n o m o u s r o a d f o l l o w i n g b y some researchers [51, 3 1 , 10]. I n [59] a m e t h o d is p r o p o s e d t o o p t i m i z e t h e road following fuzzy controller.. 25.

(26) 3.3. Comparison of Works. A comparative of the works i n the area has been realized i n order to p u t the work done i n context.. T a b l e 1 shows t h i s c o m p a r a t i v e i n t h e last 6 years. T h e p a r a m e t e r s. of d i f f e r e n t i a t i o n a r e , t h e t y p e o f v i s i o n u s e d , i f t h e w o r k i n c l u d e s d e t e c t i o n. and/or. a v o i d a n c e of o b s t a c l e s , t h e t y p e of c o n t r o l i f u s e d a n d i f t h e r e w a s a n i m p l e m e n t a t i o n of t h e s y s t e m .. T a b l e 1: C o m p a r i s o n o f t h e different w o r k s i n recent years. Year of. Autor(sJ. Title. Vision. Obstacle. Control. Implementation. Broggi, A.; Caraffi,. Obstacle Detection with. Stereo. Detection. None. Yes. C ; Fedriga, R.Lj. Stereo Vision for OIT-Road. Grisleri, P.. Vehicle Navigation. Seung-Hun K i m ;. A H y b r i d Autonomous /. Chi-Won R o h ;. Teleoperated Strategy for. Sung-Chul K a n g ;. Reiiable Mobile Robot Stereo. None. None. No. Lasser. None. None. Yes. Detection. None. Yes. Mono. Detection. None. Yes. Stereo. Detection. None. Yes. None. None. Yes. Mono. None. Neural Classifier.. Yes. Stereo. Detection. None. Yes. Yes. Publication 2005 [19]. 2006 [45]. 2006 [24]. Mono. Min-Yong P a r k ;. Outdoor Navigation. Cabani, I.;. A Fast a n d Self-adaptive Color. Toulminet, G.;. Stereo Vision Matching. Bensrhair, A. 2006 [43]. Zezhong Xu;. Obstacle Detection and Road. Yanbin Zhuang;. Following using Laser Scanner. Huahua C h e n ; 2006 [44]. PerroUaz, M . ;. Long Range Obstacle. Laser/. Labayrade, R.;. Detection Using Laser Scanner. Stereo. Royere, C . ;. and Stereovision. Hautiere, N . ; Aubert, D . ; 2007 [22]. DubbeJman. Obstacle Detection during Day and Night Conditions using Stereo Vision. 2007 [20]. 2008 [50]. van d e r M a r k , W . j. Stereo based Obstacle. van den Heuvel,. Detection with Uncertainty in. J.C.; G r o e n , F . C A ;. Rough Terrain. Hong, D . ; Kimrael,. Development of a semi-. Stereo/. S.; FJoehling, R. j. autonomous vehicle operable. Laser/GPS. Caraoriano, N. j. by the visually-impaired. Cardwell, W . ; Jannaman, G . ; Purcellj A . ; Ross, D . ; Russel, E . ; 2008 [48]. Neagoe, V . ;. Road following for. Tudoran, C.;. autonomous vehicle navigation using a concurrent neural classifier. 2009 [21]. Tiberiu Marita. Barriers Detection Method for Stereovision-Based ACC Systems. 2009 [46]. Lid oris, G . ;. The Autonomous City. Rohrmuller, F . ;. Explorer (ACE) project —. Wollhen\D ;. mobile robot navigation in. Buss, M , ;. highly populated urban. r. Laser. Detection/. Behavior. Avoidance. Selection,. Mono. None. Fuzzy. Yes. Stereo. None. Follower. Yes. Mono. None. Fuzzy. Yes. Detection/. Fuzzy. Yes. environments 2009 [9]. Yi Fu; L i , H,; Kaye,. Design and Stability Analysis. M.. of A Fuzzy Controller for Autonomous Road Following. 2010 [49]. Das, A.;. Robust visual path following. Naroditsky, 0 . ;. for heterogeneous mobile. Zhiwei Z h u ;. platforms. Samarasekera, S . ; Kumar, R.; 2010 [47]. Yi Fu ; Li, H . ;. Hardware/Software. Kaye, M . E . ;. for a Fuzzy Autonomous. Codesign. Road-Following System 2010. Hernandez,. Stereovision Feedback and. Aristeo.. Fuzzy Control for Autonomous. Stereo. Avoidance. Robot. Navigation-. 26.

(27) It c a n be c o n c l u d e d t h a t t h e r e has b e e n a l o t of r e s e a r c h i n s t e r e o v i s i o n i n recent y e a r s , a l t h o u g h t h i s w o r k of thesis uses s t e r e o v i s i o n , i t also i n c l u d e s d e t e c t i o n a n d a v o i d a n c e of o b s t a c l e s , w h i c h differentiates i t f r o m m o s t of p r e v i o u s w o r k . T h e t y p e of c o n t r o l t o be u s e d is i n f u z z y l o g i c , w h i c h n o t m a n y w o r k s select t h i s a n d t h e w o r k is i m p l e m e n t e d i n o r d e r t o be t e s t e d a n d m a k e e x p e r i m e n t s . A s far as we k n o w a c c o r d i n g t o t h e c o m p a r i s o n of t h e w o r k , i t is different t o w h a t i t has b e e n d o n e i n recent years a n d also t h e t e c h n i q u e s u s e d i n stereo v i s i o n are different f r o m t h e c l a s s i c a l ones, a n d we t h i n k , i t is a m a j o r c o n t r i b u t i o n .. 27.

(28) 4Proposal I n t h i s c h a p t e r , t h e p r o p o s a l for s o l u t i o n is p r e s e n t e d a n d t h e a l g o r i t h m t h a t w a s u s e d i n o r d e r t o detect t h e r o a d , as w e l l as t h e m e t h o d for c o n t r o l i n g t h e c a r ' s d i r e c t i o n w i t h t h e f u z z y c o n t r o l l e r a n d t h e o b s t a c l e d e t e c t i o n t o s t o p t h e v e h i c l e . T h e reasons for u s i n g e a c h m e t h o d are e x p l a i n e d a n d s o m e of t h e m , c o m p a r e d t o o t h e r s i n o r d e r t o s h o w b e t t e r p e r f o r m a n c e . T h e s o l u t i o n t o solve t h e p r o b l e m of t h i s t h e s i s is p r e s e n t e d i n t h i s c h a p t e r . F i g u r e 6 presents t h e i n d i v i d u a l c o m p o n e n t s t o solve t h e p r o b l e m .. F i g u r e 6: I n t e g r a l s o l u t i o n. 4.1. Stereovision system. T h i s p a r t of t h e p r o j e c t is f u n d a m e n t a l , b e c a u s e t h e s t e r e o c a m e r a is t h e sensor of t h e r o b o t a n d t h e efficiency of t h e t o t a l s y s t e m d e p e n d s o n t h e r e c o g n i t i o n of t h e r o a d a n d t h e i m a g e p r o c e s s i n g ; t h e o u t p u t of t h i s process is t h e i n p u t of t h e f u z z y c o n t r o l l e r . T h e s t e r e o v i s i o n s y s t e m c o n s i s t s of t w o w e b c a m c a m e r a s t h a t are a l i g n e d t o be c o n s i d e r e d as a s t e r e o c a m e r a . A stereo c a m e r a g r a b s v i d e o of a c e r t a i n scene b u t theres a n offset b e t w e e n t h e t w o i m a g e s because of t h e different p l a c e m e n t of e a c h c a m e r a . T h i s offset is c a l i b r a t e d a n d t h e n c o r r e l a t e d b e t w e e n t h e t w o c a m e r a s t o k n o w t h e d i s t a n c e of a n object i n the image. 4.1.1. Image acquisition. T h e i m a g e is a c q u i r e d b y t w o c a m e r a s p l a c e d i n front of t h e c a r . E a c h c a m e r a a c q u i r e s t h e segment of t h e r o a d i n front of t h e r o b o t , b u t w i t h a n offset b e t w e e n t h e m .. The. g r a b a c q u i s i t i o n is c a l i b r a t e d so t h a t e a c h f r a m e i n e a c h c a m e r a is t a k e n at t h e same t i m e t o process t h e i m a g e c o r r e c t l y . T h e i n t e r f a c e w i t h t h e c a m e r a s is t h r o u g h U S B .. 28.

(29) 4.1.2. Image. processing. A f t e r a c q u i r i n g t h e t w o i m a g e s , t h e y n e e d t o b e p r o c e s s e d t o get t h e n e c e s s a r y i n f o r m a t i o n t o r e c o g n i z e t h e r o a d . F i r s t , a t h r e s h o l d is a p p l i e d t o t h e o r i g i n a l i m a g e i n o r d e r t o e l i m i n a t e noise a n d e m p h a s i z e t h e r o a d . T h e n , b r i g h t n e s s , c o n t r a s t a n d g a m m a are e q u a l i z e d a c c o r d i n g t o t h e scene t o h a v e a b e t t e r r e c o g n i t i o n of t h e r o a d l i n e s . H a v i n g p r o c e s s e d t h e i m a g e , t h e edge d e t e c t i o n is d o n e , t h r e e lines are c o n s t a n t l y a n a l y z e d t o detect t h e edges. W i t h t h e i n f o r m a t i o n of t h e edges d e t e c t e d , t h e p o i n t s a, b a n d c are c a l c u l a t e d , w h i c h are t h e p a r a m e t e r s t o c a l c u l a t e 9 a n d p.. F i g u r e 7 shows t h i s. procedure.. F i g u r e 7: F l o w c h a r t of t h e i m a g e p r o c e s s i n g . A n a l g o r i t h m d e v e l o p e d was i m p l e m e n t e d t o detect t h e r o a d . B a s i c a l l y , t h e r e are f o u r h o r i z o n t a l lines t h a t are c o n s t a n t l y c h e c k e d i n e a c h i m a g e t o d e t e r m i n e t h e edges. T h e far t o p l i n e is t o detect o b j e c t s i n t h e l o n g d i s t a n c e i n o r d e r t o react a g a i n s t t h e m p r o p e r l y because t h i s l i n e is t h e one t h a t detects first t h e o b j e c t s t h a t pass t r o u g h i t . T h e o t h e r t h r e e lines are u s e d t o detect t h e r o a d itself.. T h e y are c a l l e d T o p L i n e ,. C e n t e r L i n e a n d B o t t o m L i n e , a n d t h e r e a s o n t o use t h r e e lines is b e c a u s e t h e y are t h e fewest n e c e s s a r y t o k n o w t h e angle. w h i c h d e t e r m i n e s t h e a n g l e of c u r v a t u r e of t h e. r o a d , a n d fewer i m a g e p r o c e s s i n g m e a n s faster c o m p u t e r p r o c e s s i n g . W h e n d e t e c t i n g t h e edges, 4 p o i n t s are d e t e c t e d for e a c h l i n e a n d t h e o b j e c t i v e is t o get t h e p o i n t s a, b a n d c w h i c h are i n t h e m e d i u m of t h e l i n e segments. follows,. 29. T h i s p o i n t s are c a l c u l a t e d as.

(30) The points a, y. always checked.. b. y. and c. y. Points a, x. are c o n s t a n t , b e c a u s e t h e y are fixed c o o r d i n a t e s t h a t are b. x. and c. x. t h a t represent t h e center p o i n t of t h e r o a d , are. u s e d t o c a l c u l a t e t h e slopes b e t w e e n a-b, a n d b-c.. T h e slopes are c o n v e r t e d t o angles. w h e r e a v e r t i c a l l i n e represents 0 ° . T h e difference i n angles b e t w e e n a-b a n d b-c shows t h e c u r v a t u r e of t h e r o a d . Slopes are c a l c u l a t e d as f o l l o w s ,. A n d t o k n o w t h e angle 9 R , w h i c h d e t e r m i n e s t h e angle of c u r v a t u r e of t h e r o a d ,. T h e s e t p o i n t w i l l a l w a y s be OR. T h e a n g l e of t h e v e h i c l e w i t h respect t o t h e r o a d is 0. v. w h i c h is d e t e r m i n e d b e t w e e n t h e angle of t h e l i n e b-c a n d t h e v e r t i c a l l i n e . It c a n. b e p o s i t i v e or n e g a t i v e , a c c o r d i n g t o t h e o r i e n t a t i o n of t h e v e h i c l e w i t h respect t o t h e road. p is c a l c u l a t e d as t h e p o i n t c . x. T h i s way, i t is a r e l a t i v e d i s t a n c e i n s t e a d of a. fixed. d i s t a n c e t o t h e o r i g i n of t h e i m a g e b e c a u s e t h e p o s i t i o n i n g d e p e n d s o n t h e o r i e n t a t i o n of t h e v e h i c l e a c c o r d i n g t o t h e r o a d at t h a t p o i n t .. T h e s e t p o i n t for p is a l w a y s 320,. w h i c h is t h e center of t h e x - a x i s b e c a u s e of t h e r e s o l u t i o n of t h e w e b c a m F i g u r e 8 shows t h e p o i n t s d e t e c t e d a n d c a l c u l a t e d w i t h t h i s m e t h o d .. 30. (480x640)..

(31) F i g u r e 8: P o i n t s d e t e c t e d a n d c a l c u l a t e d i n t h e i m a g e p r o c e s s i n g W h e n u s i n g a stereo c a m e r a , t h e p a r a m e t e r s for b o t h f r a m e s are c a l c u l a t e d as t h e average v a l u e of e a c h p a r a m e t e r .. 4.2. Fuzzy Controller. T h e f u z z y c o n t r o l l e r is t h e b r a i n of t h e r o b o t a n d t h e p r o c e s s i n g d o n e d e t e r m i n e s t h e m a n i p u l a t i o n d o n e t o t h e a c t u a t o r s i n o r d e r t o keep t h e v e h i c l e i n t h e d e s i r e d p o s i t i o n . T h e i n p u t s of t h e c o n t r o l l e r are •& a n d p, w h i c h are c a l c u l a t e d as s h o w n i n s e c t i o n 4.1 after t h e i m a g e p r o c e s s i n g a n d r o a d r e c o g n i t i o n . I n t h e present w o r k , a f u z z y c o n t r o l l e r was d e v e l o p e d t o keep t h e c a r i n s i d e t h e r o a d a n d f o u r l i n g u i s t i c v a r i a b l e s were u s e d : t h e angle of t h e c a r r e l a t i v e t o t h e angle of t h e r o a d 9 a n d t h e d i s t a n c e f r o m t h e c a r ' s center t o t h e r o a d center l i n e p.. The. l i n g u i s t i c e x p r e s s i o n s u s e d t o d e s c r i b e t h e m a g n i t u d e of t h e l i n g u i s t i c v a r i a b l e s c o n t a i n the following basic adjectives: negative very b i g ( - V B ) , negative b i g (-B), negative s m a l l (-S), zero (o), p o s i t i v e s m a l l ( + S ) ,. positive big (+B). and positive very big. (+VB).. T h i s m e a n s t h a t t h e set of e v e r y l i n g u i s t i c v a r i a b l e c o n t a i n s as m a n y e l e m e n t s as t h e n u m b e r of a d j e c t i v e s u s e d t o d e s c r i b e t h e v a r i a b l e . T h e c a r d i n a l i t y of t h i s set denotes t h e n u m b e r of e l e m e n t s i n t h i s set. defuzzification procedures.. T h e s e sets are t h e basis for t h e f u z z i f i c a t i o n a n d. S i n c e n o e x p e r t k n o w l e d g e was a v a i l a b l e , t h e r u l e base. p r o p o s e d b y [36] was i m p l e m e n t e d u s i n g t h e a b o v e b a s i c a d j e c t i v e s . A c c o r d i n g t o [23], t h e M a n d a n i i m p l i c a t i o n is t h e m o s t i m p o r t a n t i m p l i c a t i o n ( I F - T H E N r u l e ) k n o w n i n f u z z y c o n t r o l l i t e r a t u r e . T h i s fact e x p l a i n s w h y t h e M a n d a n i i m p l i c a t i o n was e m p l o y e d. 31.

(32) i n t h e present. development.. A c c o r d i n g t o [23], t h e c o n t r o l r u l e s are d e s i g n e d b a s e d o n e x p e r t k n o w l e d g e testing.. F u r t h e r m o r e , t h e c o n t r o l rules also meet t h e s t a b i l i t y r e q u i r e m e n t s. f r o m L y a p u n o v ' s direct method.. and. derived. F o r e x a m p l e , i f p is + B a n d is i n c r e a s i n g r a p i d l y 9,. t h e n t h e v e h i c l e s h o u l d t u r n left, i.e. 9. S. s h o u l d b e + B . B a s e d o n t h i s k n o w l e d g e , we c a n. o b t a i n t w e n t y five f u z z y r u l e s . T a b l e 2 represents a b s t r a c t k n o w l e d g e t h a t a n e x p e r t uses t o c o n t r o l t h e s t e e r i n g angle g i v e n f r o m i n f o r m a t i o n a b o u t t h e e r r o r of 9 a n d p . T h e i n p u t a n d o u t p u t l i n g u i s t i c v a r i a b l e s are s u m m a r i z e d i n t h e t a b l e .. T a b l e 2: Set of r u l e s of t h e f u z z y c o n t r o l l e r .. ++ + 0. --. ++. +. P 0. -VB. -VB. -B. -VB. -B. -B -B. -S -S. -S 0 +s. -S. -S. +B. -. --. +s +s +s. +s. +B. +VB. +VB. +VB. +B +B. N o t e : V B : V e r y B i g ; B : B i g ; S: S m a l l ; 0: Z e r o . I n o r d e r t o a p p l y t h e f u z z y c o n t r o l , 9 a n d p n e e d t o be d e t e r m i n e d ; these are c a l c u l a t e d i n t h e i m a g e p r o c e s s i n g p a r t a n d are t h e i n p u t for t h e f u z z y c o n t r o l . F u z z i f i c a t i o n n e e d t o b e d o n e i n o r d e r t o m a p t h e i n p u t s f r o m c r i s p values t o grades of. member¬. s h i p for l i n g u i s t i c t e r m s of f u z z y sets a n d t h e n a c c o r d i n g t o t h e r u l e base, a degree of t r u t h is c a l c u l a t e d ; t h e d e f u z z i f i c a t i o n process m a p s t h e values t o r e a l values for t h e m a n i p u l a t i o n . T h e process for t h e f u z z y c o n t r o l l e r is s h o w n i n figure 9.. 32.

(33) F i g u r e 9: F l o w c h a r t of t h e f u z z y c o n t r o l l e r . A b l o c k d i a g r a m of t h e f u z z y c o n t r o l l e r is s h o w n i n F i g . 10. T h e d e s i r e d o r i e n t a t i o n of t h e center l i n e of t h e c a r s h o u l d be a l i g n e d w i t h t h e r o a d c e n t r o i d . T h e e r r o r is t h e a n g l e b e t w e e n t h e d e s i r e d o r i e n t a t i o n of t h e center l i n e a n d t h e a c t u a l center l i n e of t h e car. T o r e d u c e t h e e r r o r t o z e r o , t h e s t e e r i n g a n g l e s h o u l d b e e q u a l t o t h e a n g l e of t h e road.. T h e f u z z y c o n t r o l l e r is d o n e i n a p r o g r a m i n L a b V I E W w h i c h i t first does t h e. f u z z i f i c a t i o n t o t h e i n p u t s , a n d w i t h t h e r u l e base i n T a b l e 2 t h e inference m e c h a n i s m is a p p l i e d . T h e d e f u z z i f i c a t i o n is a p p l i e d t o t h o s e r e s u l t s f r o m t h e inference m e c h a n i s m a n d i t is sent t o t h e a c t u a t o r s of t h e v e h i c l e t o keep i t i n t h e d e s i r e d p o s i t i o n .. 33.

(34) F i g u r e 10: B l o c k d i a g r a m of t h e f u z z y l o g i c c o n t r o l l e r . T h e f u z z i f i c a t i o n p r o c e d u r e m a p s t h e c r i s p i n p u t values t o t h e l i n g u i s t i c f u z z y t e r m s w i t h t h e m e m b e r s h i p values b e t w e e n 0 a n d 1. T h e f u z z i f i c a t i o n m e t h o d u s e d is m a x m i n . I n t h i s t h e s i s , we use five m e m b e r s h i p f u n c t i o n s for b o t h e r r o r i n p a n d e r r o r i n 9. F i g u r e s 11 a n d 12 i l l u s t r a t e t h e n o r m a l i z e d i n p u t m e m b e r s h i p f u n c t i o n s for p a n d 9 respectively.. F i g u r e 11: I n p u t m e m b e r s h i p f u n c t i o n s for e r r o r i n p.. 34.

(35) F i g u r e 12: I n p u t m e m b e r s h i p f u n c t i o n s for e r r o r i n 9.. T h e d e f u z z i f i c a t i o n p r o c e d u r e m a p s t h e f u z z y o u t p u t f r o m t h e inference m e c h a n i s m t o a c r i s p s i g n a l . It is u s e d C e n t e r of S u m s as t h e d e f u z z i f i c a t i o n m e t h o d t o c o m b i n e t h e r e c o m m e n d a t i o n s r e p r e s e n t e d b y t h e i m p l i e d f u z z y sets f r o m a l l t h e r u l e s . F i g . 13 shows the normalized output membership functions.. I n t h i s case, for faster c o m p u t a t i o n a l. s p e e d , t h e t y p e of m e m b e r s h i p f u n c t i o n s are c a l l e d s i n g l e t o n s , e a c h one has t h e v a l u e of 1.. F i g u r e 13: O u t p u t m e m b e r s h i p f u n c t i o n s .. 35.

(36) 4.3. Software Plataform. I n o r d e r t o d e v e l o p t h e code t o i n t e g r a t e t h e s t e r e o v i s i o n s y s t e m a n d t h e f u z z y c o n t r o l l e r w i t h t h e h a r d w a r e , t h e use of t h e p r o g r a m L a b V I E W is p r o p o s e d m a i n l y b e c a u s e i t s h a r d w a r e i n t e g r a t i o n a n d t h e a b i l i t y t o g r a b i m a g e s f r o m t w o c a m e r a s of t h e. same. m o d e l t h r o u g h U S B . T h e d a t a d i s p l a y , c u s t o m c o n t r o l s a n d t h e user i n t e r f a c e are also aspects a n a l y z e d t o choose t h i s p l a t a f o r m for d e s i g n i n g . A p p e n d i x B shows m o r e a b o u t t h i s software p l a t f o r m a n d t h e a c t u a l c o d e t o i m p l e m e n t t h i s p r o p o s a l .. 4.4. Hardware. T o i m p l e m e n t t h i s thesis a n d t o c a r r y o u t t h e e x p e r i m e n t s , t h e s t r u c t u r e of a n R / C c a r is p r o p o s e d .. It has de b a s i c c o m p o n e n t s. desired: an steering system a n d a motor. for t h e t r a c t i o n . T h e o t h e r e l e m e n t s of t h e c a r are n o t u s e d ; i n s t e a d , e l e m e n t s w o u l d b e a d d e d t o c o m p l e t e t h e o b j e c t i v e . T h i s c o m p o n e n t s w o u l d be e x p l a i n e d i n c h a p t e r 5, a n d t h e s p e c i f i c a t i o n s of t h e h a r d w a r e is s h o w n i n A p p e n d i x A .. 36.

(37) 5. E x p e r i m e n t s a n d Results. I n t h i s c h a p t e r , a l l t h e e x p e r i m e n t s d o n e i n o r d e r t o p r o v e t h e p r o p o s a l are p r e s e n t e d . A l s o , t h e c o m p o n e n t s u s e d , a n d t h e i n f r a e s t r u c t u r e of t h e p r o j e c t are s h o w n . I n t a b l e 3, a list of t h e e x p e r i m e n t s d o n e a n d p r e s e n t e d i n t h i s s e c t i o n is e x p l a i n e d .. T a b l e 3: E x p e r i m e n t s c a r r i e d o u t Experiment. Objective. Road Detection. T o r e c o g n i z e t h e r o a d a c c u r a t e l y i n different s i t u a t i o n s .. Distance Calculation Object Detection. different p o i n t s a c c u r a t e l y . T o detect o b j e c t s present o n t h e surface of t h e r o a d . T o k n o w t h a t t h e c a r m a i n t a i n s i t s p o s i t i o n across t h e. Controllability. 5.1. T o calculate the distance between the camera a n d the. road.. Hardware. T h e r o b o t was b u i l t u s i n g a r e m o t e c o n t r o l c a r as a base, b u t t h e s t r u c t u r e was m o d i f i e d i n o r d e r t o leave space for t h e a c t u a t o r s a n d sensors. B a s i c a l l y , o n l y t h e s t e e r i n g p a r t , a n d t h e wheels of t h e r e m o t e c o n t r o l c a r were u s e d . figure. T h e b a s i c s t r u c t u r e is s h o w n i n. 14.. F i g u r e 14: B a s i c s t r u c t u r e of t h e m o b i l e r o b o t .. 37.

(38) 5.1.1. Actuators. T h e m a i n a c t u a t o r s of t h e m o b i l e r o b o t are t w o s e r v o m o t o r s a n d one D C m o t o r .. One. s e r v o m o t o r is a t t a c h e d t o t h e s t e e r i n g s y s t e m , i n o r d e r t o m o d i f y t h e s t e e r i n g a n g l e a c c o r d i n g t o t h e f u z z y c o n t r o l l e r o u t p u t . I n figure 15 t h e s e r v o m o t o r is s h o w n w i t h t h e steering system.. F i g u r e 15: S e r v o m o t o r a t t a c h e d t o t h e s t e e r i n g s y s t e m . T h e o t h e r s e r v o m o t o r is u s e d t o m o d i f y t h e s p e e d of t h e c a r . T h i s s e r v o m o t o r is c o n t r o l l e d t o m o v e a c e r t a i n angle; t h e s e r v o m o t o r is c o u p l e d t o a p o t e n t i o m e t e r t h a t r e g u l a t e s t h e i n p u t v o l t a g e of t h e m i c r o c o n t r o l l e r , w h i c h c o n v e r t s i t i n t o a P W M s i g n a l t h a t r e g u l a t e s t h e v o l t a g e of t h e D C m o t o r i n o r d e r t o s p e e d i t u p or d o w n . F i g u r e 16 shows t h e s c h e m a t i c d i a g r a m of t h i s c o n f i g u r a t i o n .. F i g u r e 16: S y s t e m t h a t r e g u l a t e s t h e s p e e d of t h e r o b o t . T h e s e r v o m o t o r s are c o n t r o l l e d u s i n g a S e r v o m o t o r C o n t r o l l e r C a r d m a d e b y L y n x 38.

Figure

Figure 2. The proposed algorithm exploits all the information available in a single image to detect the drivable road surface ahead the target vehicle.

Referencias

Documento similar

In our approach, we retain the idea of using both mathematical programming and fuzzy membership to detect overlapping communities, but we replace the fuzzy objective function

Keywords: iPSCs; induced pluripotent stem cells; clinics; clinical trial; drug screening; personalized medicine; regenerative medicine.. The Evolution of

Astrometric and photometric star cata- logues derived from the ESA HIPPARCOS Space Astrometry Mission.

The photometry of the 236 238 objects detected in the reference images was grouped into the reference catalog (Table 3) 5 , which contains the object identifier, the right

In addition to traffic and noise exposure data, the calculation method requires the following inputs: noise costs per day per person exposed to road traffic

In the edition where the SPOC materials were available, we have observed a correlation between the students’ final marks and their percentage rate of

Cu 1.75 S identified, using X-ray diffraction analysis, products from the pretreatment of the chalcocite mineral with 30 kg/t H 2 SO 4 , 40 kg/t NaCl and 7 days of curing time at

Authors: Pedro Manuel Moreno-Marcos, Dánae Martínez de la Torre, Gabriel González Castro, Pedro J.. Muñoz-Merino and Carlos