VANET performance evaluation heavily depends on conducting simulations [288]. Real world experiments have been performed, but achieved density of vehicles equipped with VANET tech- nology is limited. Moreover, it is hard to reproduce scenarios with many involved vehicles in practice [285]. Hence, most studies on VANET performance rely on simulation results.
A common feature of discrete event simulators, like the ones used for VANETs, is that they do not take computational requirements encountered at individual nodes into regard to evaluate system performance. This means that the obtained results represent a system in which processing of data at the individual nodes takes zero time. This is caused by the behavior that a discrete time stamp increase only happens after all processing started at the prior time stamp has ended [265, 309]. However, in reality time clearly proceeds, while computational operations are ongoing. Thus, one has to make sure that used algorithms are fast enough for practical usage in a separate evaluation step.
framework focus missing standardized features traffic flow simulator
network simulator Veins [288] WAVE security, ASN.1, time/position
coupling
SUMO OMNET++
Artery [262] ETSI ITS time/position coupling SUMO OMNET++
no ITS-G5 (802.11p used in- stead), DCC
iTETRIS [155, 215]
ETSI ITS security, ASN.1, time/position coupling
SUMO ns-3
VANETSim [302] security no standard compliance own own VNS [137] traffic flow no standard compliance own ns-3,
OMNET++
VSimRTI [277] ETSI ITS ? SUMO,
VISSIM
ns-3, OMNET++
ezCar2X [266] ETSI ITS - SUMO ns-3
EstiNet [318] SDN security, all above network layer
own own
Table 2.1: Comparison of features of different simulation frameworks for VANETs
Available simulation environments for VANET evaluation are looked at in more detail in Section 2.4.1. Moreover, methods for measuring the computational performance of algorithms are presented in Section 2.4.2.
2.4.1 VANET Simulation Environments
Several simulation environments for VANETs have been developed [137, 215, 266, 277, 288, 302, 318]1. These vary greatly in regard to standard conformance and the set of supported features. Typically, a combination of different tools for simulation of traffic flow, communication network and protocol behavior is used. This follows the tool coupling approach proposed for VANET simulation in [278]. Popular network simulators are ns-3 and OMNET++ [263, 309]. Microscopic traffic flow simulation is often provided by Simulation of Urban MObility (SUMO) [18]. The VSimRTI framework can also utilize VISSIM [136] for traffic flow simulation. An overview of the features provided by the different frameworks is given in Table 2.1. A similar discussion is provided in the author’s prior work given in [43]2.
All of the frameworks from Table 2.1, except of the last three ones (VSimRTI, ezCar2X, Es- tiNet), are freely available in open source form. Work on further frameworks TraNS, GrooveNet, NCTUns (predecessor of commercial EstiNet) and MobiREAL, all looked at in [88, 220], has 1The author’s contribution to [266] mainly relates to the design and implementation of security functionality, platform independent data encoding schemes and integration into ns-3. The remaining contributions are from the coauthors.
2
The coauthor’s contribution to [43] mainly relates to the implementation of functionality enabling the conducted tests of the ezCar2X framework. The main contribution is from the author of this work.
been discontinued. Thus, they are not taken into regard in this work. Features of VSimRTI are gathered from the publicly available information about this tool set, as studying the source code is not possible [59, 76, 277].
The Artery framework’s feature support is close to the one of ezCar2X. However, due to the usage of 802.11p on the physical and MAC layers no ETSI ITS conforming DCC can be realized. DCC requires channel usage information from the MAC layer [103], which is not available in the Artery approach. Moreover, this also means that the strict MAC layer message size limit from ETSI ITS is not enforced by Artery.
The simulation framework utilized in this work is based on the ezCar2X framework [266, 267]3. An in detail description and a comparison to other simulation approaches is provided in Section 3.3.
2.4.2 Computational Performance Measurement
Computational performance can be benchmarked with different performance metrics [153, 219]. Important metrics used in this work are runtime and memory footprint (stack and heap). A methodology for accurate time measurement avoiding unintended influence of modern proces- sors out-of-order execution is provided in [246]. Following this methodology, a so called seri- alization instruction is additionally inserted before and after the code fragment whose runtime is to be measured. Available serialization instructions are processor dependent, e.g., for all Intel processors the cpuid instruction can be used [246].
All runtime measurement results provided in this work have been collected using the outlined strategy from [246]. More details about the applied computational performance measurement methodology are given in Section 3.4.
2.4.3 Traffic Scenarios
Many different traffic scenarios have been suggested for usage in the evaluation process of VANETs. A traffic scenarios consists of the road topology and the traffic flow built up by mobile nodes traveling on the road topology. Optionally, RSUs may be present in the scenario, too. Popular road topologies include
• highway [52, 56, 93, 106, 190], • highway crossing [93], • rural road [204],
• urban grid [56, 93, 288, 304], and • urban roundabout [93].
The highway crossing scenario provides high traffic density, but mutual influence between the individual highways is identified as low in [52]. Hence, we do not consider this setup as a separate evaluation scenario. The detailed parameters of scenarios used throughout this work are given in Section 3.2.
3For remarks on [266] see footnote 1.