TÍTULO NOVENO

To overcome the difficulties of obtaining a proprietary simulator and the associated learning curve, limited interfaces, and the complexities & time required to quantify all of the various parameters for the scaled vehicle, a simple 2D graphical real- time hardware-in-the-loop simulator was developed for this project. The simulator was created to aid in algorithm development for a 1/10th scale model collision avoidance system that will be used to study human factors related to this technology and is based on the bicycle model as described in chapter 2. A custom simulator provides more control over the interface and the underlying vehicle dynamics while avoiding the black-box ambiguity of commercial simulators. In addition, the vehicle’s sensor could be simulated more easily. The goal while developing the simulator was to strike a balance between complexity and usability that still captures all the important vehicle dynamics most pertinent to developing and testing collision avoidance algorithms. A 2D graphical simulation, developed in the Java programming language, suffices to study these

algorithms and human factors as there is no need to use a high-fidelity 3D simulator. Use of pre-existing software libraries were incorporated into this project, when available, to improve productivity and reliability, such as to obtain driver input from the force- feedback joystick. By using the Java programming language and open-source API’s the simulator is developed to be platform independent, highly customizable, and easily understandable, especially if future users see a need for change; this creates a very low- cost hardware-in-the-loop simulator for testing and development. A 2D simulation also

109

drastically reduces the computational overhead when compared to a 3D simulation which makes real-time operation much easier to achieve on average computing hardware; a 2D simulation also lends itself naturally to the incorporation of the bicycle model which ignores the 3D vehicle dynamic effects, including roll and pitch. The human and hardware-in-the-loop simulation runs the same closed-loop software on the same

hardware as when the system runs on the ground. This keeps the hardware limitations in perspective throughout development and mitigates the additional effort associated with translating the hardware and software to an experimental scaled vehicle platform for testing. The testing environment can be switched seamlessly. This allows for fast

deployment between software updates and gives quick comparisons between the software running in an idealized environment with perfect sensors and to reality with real

hardware and sensor constraints. Overall, the simulation reduces the expensive and time- consuming experimental testing and provides verification of the controller before ground testing, which reduces safety concerns, especially in preliminary stages of development, and provides further means to test and design a semi-autonomous collision avoidance system.

As discussed by Milliken, a useful concept when approaching difficult subject matter such as vehicle dynamics is the Ladder of Abstraction. This concept applies well to the design considerations associated with the development of a computer simulation designed to capture a vehicle’s dynamic behavior, to simulate a vehicle’s sensor(s), and to evaluate a controller running ADAS algorithms. The ultimate reality of a vehicle’s dynamics is only had by experiencing it and anything less is an abstraction or an

110

be had by using the controller’s output. This complete reality is shown at the top and is composed of the functioning vehicle with a human driver behind the wheel and the vehicle’s controller intervening when the threat of an imminent collision is detected. The move down the ladder occurs by applying simplifying assumptions, linearizing equations, and simulating sensor & hardware components. The simulation utilizes the bicycle model to simplify the vehicle dynamics to estimate the motions of the vehicle and is suitable for testing autonomous collision avoidance algorithms. The vehicle’s embedded controller running the ADAS algorithms is kept in-the-loop by passing it the simulated data and using its output; this moves the results further up the ladder and closer to a complete reality. In addition, a dynamometer was built to allow the vehicle to accelerate and turn while running the simulation on a desk; this also brings the results closer to a complete reality as the actual motor’s response and measured wheel speeds can be used within the simulation. The dynamometer is described in chapter 5. The delineation between reality and simulation is shown in Figure 50; the motor dynamics and wheel encoder data is drawn on the line between reality and abstraction as both can either be simulated or measured when the vehicle is used with the dynamometer. A description of how the simulation operates follows.

111

Figure 50. Delineation of Reality and Simulation

The simulation serves to create a virtual environment with obstacles that a

simulated LiDAR sensor captures as a point-cloud and that a driver may navigate through with standard steering wheel and pedal inputs. This configuration allows the embedded controller to use the exact same code whether it is actually running on the ground or whether it is running through a simulated environment and provides seamless transition between environments. The simulation passes the simulated LiDAR sensor’s point-cloud to the embedded controller via serial communication, which is facilitated by an FTDI breakout board. The embedded controller then operates as if it were on the ground by running the ADAS algorithms and generating a corrective steer response if the threat of an imminent collision is determined. This entails processing the simulated point-cloud by segmenting the data and parsing out road-cones and obstacles. The segmentation results

112

are then passed through the RRT path-planning algorithm and used to calculate the required steer angle to avoid a collision. The calculated steer angle is passed back to the simulation to update the vehicle’s dynamics and to observe the response. The serial connection allows the simulation code and the controller code to be completely separate and modular; only the communication protocol between the simulation and controller must be consistent and predetermined. This allows the development of the simulation and the embedded controller to be completely independent; making changes to the

controller’s software does not require changes to the simulation nor does it affect the simulation’s operation as long as the communication protocol persists.

Just as with the embedded controller’s software, a multithreaded architecture was used to run the various tasks necessary to run the real-time simulator. A real-time

simulator refers to a computer model of a physical system that runs at the same rate as actual time. Multithreading was incorporated into the design of the simulator to improve efficiency and to meet the real-time simulation constraints. In addition, code was

carefully placed to synchronize operation. For example, after the vehicle passes the simulated sensor data to the vehicle’s controller it must wait for a response from the controller. While waiting for the vehicle controller to process the data and respond, the simulation performs other tasks associated with updating the vehicle and rendering the graphics. The core of the simulation takes place on three threads: the event dispatch thread, the update thread, and the render thread. The Event Dispatch Thread (EDT) handles all the events associated with the GUI components such as: button presses, checkboxes, radio buttons, etc. The update thread handles updating the simulation physics. The render thread handles drawing the simulation and its objects. Key bindings

113

were used to process user input from the keyboard. Each class has its own draw and update methods, when applicable, which are called from the render and update thread, respectively. Data is shared across the simulation by using a static data class or by passing references. A core class within the simulation is the car class. With respect to the update thread, this is where the bicycle model based vehicle’s physics are implemented, as discussed in chapter 2. This class handles updating all the physics for each instance of the car object. For the driver’s vehicle, this class also handles simulating the car’s LiDAR sensor scans. The simulated LiDAR sensor captures the point-cloud data of the simulated environment from the car’s frame of reference. This is done by sweeping lines across the driving plane and searching for intersections. A line is generated using the Line2D object which originates at the car and spans the sensor’s range. The line is searched for

intersections with other vehicles and cone objects and the nearest intersection is added as a point to the point-cloud. The simulated sensor’s range, resolution, and scan frequency can be adjusted via the GUI and the controller’s software permits a point-cloud composed of any number of points. Within the update thread, after a simulated scan the point-cloud data is buffered in the serial communication class and a sub thread is run to pass the simulated LiDAR data to the controller hardware-in-the-loop. The serial communication gets the controller’s response and buffers it for the next vehicle physics update. The elegant nature of object-oriented programming (OOP) allows many cars to be simulated simultaneously by creating many instances of the car object that all have the bicycle model based physics and parameters built-in.

In order to run a simulation that constantly updates the physics of each vehicle through numerical integration of the differential equations and then renders the results in

114

a simulated environment incorporation of a programming loop is required. Game and simulation programming have a lot in common and overlap in numerous ways. Games often simulate the behavior of objects, vehicles, and collisions for the enjoyment of the player. The design patterns between a game and a simulation are very similar. The most quintessential component of this design pattern is the loop, often referred to as a game– loop. This loop is where the physics are updated, the scene is rendered, and where events are processed, such as user input. The game-loop is the heartbeat of every game or

simulation. There are many ways to implement a game/simulation loop and the individual implementation of can vary greatly from simulation to simulation (or game to game). The loop dictates when and how fast the simulation state is updated and the scene is rendered. A simple loop may have the form below in Figure 51.

Figure 51. Simple game-loop/simulation-loop

The above loop has no timing considerations and operates the rendering and physics updates in the same loop. However, both the simulation physics and rendering loop need to run at different rates while regulating the time. This is accomplished using multiple threads, one to run the simulation update loop and one to run the rendering loop. Each loop regulates its speed by using a high resolution system timer. The simulation

115

must be updated much faster than the scene needs to be rendered in order to maintain numeric stability regarding the vehicle physics. In addition, rendering faster than the monitor’s refresh rate or faster than is perceivable by the human eye is unnecessary and incurs high processor overhead. Real-time operation is realized by updating the

simulation and scene with the actual elapsed system time. An example of a time regulated loop is shown below in Figure 52.

Figure 52. Multithreaded game-loop/simulation-loop with time regulation

The use of a game loop allows for complete control over the timing of simulation and for active rendering. This is of the utmost important, especially in game and real-time

simulation development, as the distances and state of the vehicle sprite have a physical dependence on the elapsed time.