Thesis contributions

The main contributions of this dissertation are presented below:

Chapter 2: Theoretical background:

• Review the state of the art of the topics covered in this dissertation: ITS, AI, UAVs, and wireless communications technologies.

Chapter 3: Unmanned Aerial Vehicles:

• In-depth review of the state of the art of flying ad hoc networks (FANETs) composed of UAVs, focusing on mobility models, positioning protocols, propagation models, and routing protocols.

• Detailed study of the impact the UAV had on the radiation pattern of the integrated WiFi communication module using a controlled environment such as an anechoic chamber.

• Comparative study of the performance provided by different FANET routing protocols, in terms of throughput and packet loss, in a real deployment consisting of several UAV nodes using WiFi on 2.4 GHz and 5 GHz bands.

The articles related to this chapter that have been published in technical journals or international conferences are:

Guillen‐Perez, A.; Cano, M.-D., “Flying ad hoc networks: A new domain for network communications,” Sensors, vol. 18, no. 10, p. 3571, Oct 2018, doi:10.3390/s18103571

2018 Journal Impact Factor (JIF): 3.031. (Q1), Rank: 15/61 in Instrument &

Instrumentation.

Guillen-Perez, A.; Montoya, A-M; Sanchez-Aarnoutse, J. C.; Cano, M.-D., “A comparative performance evaluation of routing protocols for flying ad-hoc networks in real conditions,” Appl. Sci., vol. 11, no. 10, p. 4363, May 2021, doi:

10.3390/app11104363

2020 Journal Impact Factor (JIF): 2.679. (Q2), Rank: 38/90 in Engineering Multidisciplinary.

Guillen-Perez, A.; Sanchez-Iborra, R.; Cano, M.-D., Sanchez-Aarnoutse, J. C.;

and Garcia-Haro, J., “WiFi networks on drones,” in 2016 ITU Kaleidoscope: ICTs for a Sustainable World (ITU WT), Nov. 2016, pp. 1–8, doi: 10.1109/ITU- WT.2016.7805730.

Guillen‐Perez, A.; Cano, M.-D., “Comunicaciones Inalámbricas con Vehículos Aéreos no Tripulados”, I Jornadas Doctorales UPCT, Universidad Politécnica de Cartagena. 2018. Oral communication.

Chapter 4: Smart Cities and Pedestrians:

• Development of a novel passive WiFi-based method to estimate the number of pedestrians at an intersection to improve the performance of traffic light ITS.

• Real scenario deployment, to show the benefits that the previously proposed algorithm could offer, as well as the possible improvements that could be made.

• By using AI algorithms, an algorithm based on the previous one was obtained, which allowed obtaining a superior performance in terms of accuracy and recall. Thus, by analyzing the WiFi messages sent passively by devices carried by pedestrians, the system was able to estimate the number of pedestrians at an intersection with high accuracy.

The articles related to this chapter that have been published in technical journals or international conferences are:

Guillen‐Perez, A.; Cano, M.-D., “Pedestrian Characterization in Urban Environments Combining WiFi and AI,” Int. J. Sens. Networks, vol. 37, no. 1, p. 48, 2021, doi: 10.1504/IJSNET.2021.117964.

2020 Journal Impact Factor (JIF): 1.302. (Q4), Rank: 80/91 in Telecommunications.

Guillen‐Perez, A.; Cano, M.-D., “A WiFi-based method to count and locate pedestrians in urban traffic scenarios,” in 2018 14th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), Oct.

2018, vol. 2018-Octob, pp. 123–130, doi: 10.1109/WiMOB.2018.8589170.

Guillen‐Perez, A.; Cano, M.-D., “Counting and locating people in outdoor environments: a comparative experimental study using WiFi-based passive methods,” ITM Web Conf., vol. 24, pp. 1–10, Feb. 2019, doi:

10.1051/itmconf/20192401010.

Chapter 5: Smart Traffic Light Control – AI approach:

• Detailed study of the principle of operation of the main adaptive traffic light control systems.

• Development of an advanced adaptive traffic light control algorithm based on queuing theory. This system was optimized using genetic algorithms.

The articles related to this chapter that have been published in technical journals or international conferences are:

Guillen‐Perez, A.; Cano, M.-D., “Intelligent IoT systems for traffic management: A practical application,” IET Intell. Transp. Syst., vol. 15, no. 2, pp.

273–285, Feb. 2021, doi: 10.1049/itr2.12021.

2020 Journal Impact Factor (JIF): 2.496. (Q2), Rank: 135/273 in Engineering, Electrical & Electronic.

Guillen‐Perez, A.; Cano, M.-D., “Optimización de un Sistema Inteligente de Control y Gestión de Transporte en Intersecciones por medio de un Algoritmo Genético”, V Jornadas Doctorales UPCT, Universidad Politécnica de Cartagena.

2019. Oral communication.

Guillen‐Perez, A.; Cano, M.-D., “Influencia del ciclo de trabajo de los semáforos en una intersección simple en múltiples parámetros ante una densidad de tráfico incremental,” in XIV Jornadas de Ingeniería Telemática (JITEL 2019), 2019, no. JITEL, pp. 22–24, [Online]. Available: http://jitel2019.i3a.es/.

Chapter 6: Interoperability of Connected Autonomous Vehicles and Intelligent Transportation Systems:

• Novel development of an Autonomous Intersection Management (AIM) system using Multi-Agent Deep Reinforcement Learning (MADRL). The proposed AIM was named RAIM.

• RAIM enhancement through recurrent neural networks, as well as through other training acceleration methods such as curriculum learning for RL, Prioritized Experience Replay (PER). Thus, the proposed system called adv.RAIM, enabled much smarter vehicular control at intersections, reducing waiting time to a minimum.

• Study of the current state-of-the-art in the field of imitation learning, learning from observation and learning from demonstration, proposing a new Learning from Demonstration (LfD) algorithm for environments where there is no (or there is a hidden expert agent) from which to extract new demonstrations. The proposed system is able to train an agent by imitation that mimics the behavior of the hidden expert agent. This trained agent is called an Oracle. This Oracle is the one used by the proposed algorithm to train an agent by demonstration, speeding up vastly the training of new agents by MADRL.

• Development of a latency-aware AIM for 5G communication network. Thus, this proposed AIM was able to guarantee maximum security due to a latency forecaster module based on Transformers and to incorporate the temporal behavior of latency in the adv.RAIM control module based on MADRL.

• Proposal of the necessary modules so that the new AIM systems can be natively integrated into the future 6G communications network. In this way, vehicle control via AIM and 6G will reduce development costs and improve the performance of both.

The articles related to this chapter that have been published in technical journals (or currently under review) or international conferences are:

Guillen‐Perez, A.; Cano, M.-D., “AIM5LA: A Latency-Aware Deep Reinforcement Learning-Based Autonomous Intersection Management system for 5G Communication Networks,” Sensors, vol. Accepted, pp. 1–20, 2022.

2020 Journal Impact Factor (JIF): 3.576. (Q2), Rank: 82/273 in Engineering, Electrical & Electronics.

Guillen‐Perez, A.; Cano, M.-D., “Multi-Agent Deep Reinforcement Learning to Manage Connected Autonomous Vehicles at Tomorrow’s Intersections,” IEEE Trans. Veh. Technol., vol. On review, pp. 1–12, 2022.

2020 Journal Impact Factor (JIF): 5.978. (Q1), Rank: 15/91 in Telecommunications; (Q1), Rank: 32/273 in Engineering, Electrical & Electronic.

Guillen‐Perez, A.; Cano, M.-D., “Learning from Oracle Demonstrations – A new approach to develop Autonomous Intersection Management control algorithms based on Multi-Agent Deep Reinforcement Learning,” IEEE Access, vol.

On Review, pp. 1–12, 2022.

2020 Journal Impact Factor (JIF): 3.367. (Q2), Rank: 65/161 in Computer Science & Information Systems.

Guillen‐Perez, A.; Cano, M.-D., “6G Communications Network Framework in the context of Edge-Decentralized Cooperative Autonomous Driving,” Appl. Sci., vol. On Review, pp. 1–12, 2022.

2020 Journal Impact Factor (JIF): 2.679. (Q2), Rank: 38/90 in Engineering Multidisciplinary.

Guillen‐Perez, A.; Cano, M.-D., “Cómo la superresolución puede ayudar a los vehículos autónomos conectados”, VI Jornadas Doctorales UPCT, Universidad de Murcia. 2020. Oral communication.

Guillen‐Perez, A.; Cano, M.-D., “RAIM: Reinforced Autonomous Intersection Management - AIM based on MADRL,” in NeurIPS 2020 - Workshop Challenges of Real-World RL, 2020, pp. 1–12, Accessed: Feb. 16, 2022. [Online]. Available:

https://www.researchgate.net/publication/357957238_RAIM_Reinforced_Autono mous_Intersection_Management_-_AIM_based_on_MADRL.