8.5.4 Adaptability Metric
The final metric attempts to quantify the adaptability of a specific engagement strategy. This metric is given by A = min i Ai− T X j=1 xij , (8.4)
where Ai denotes the initial amount of ammunition available to WS i and the remaining param- eters have the same meanings as before.
The metric A is designed to measure the propensity of an engagement strategy to maximize the number of times that a WS is available for re-engagement after the proposed assignment, thereby ensuring that as many WSs as possible are reusable in future engagements. By using ammunition more effectively during an engagement, WSs on the ground will be more adaptable to changing conditions, such as responding to newly detected threats and performing follow-up engagements.
8.6 Practical Simulation Characteristics
In addition to the metrics introduced in §8.5, certain periphery characteristics also need to be considered in order to ensure a successful functioning TEWA system. These include the time required to generate WA allocation suggestions and the memory storage requirements of the internal algorithms of the system.
Scenario space saturation
An important consideration of a specific GBAD scenario setup and the constituent TEWA algorithms, are the number of threats that can be countered effectively. This characteristic is a function of both the TEWA algorithms and the specific setup (positions and numbers of DAs, WSs and threats). A simulation performance evaluation framework may be used to ascertain the scenario space saturation characteristics of a specific algorithms-scenario combination. On the contrary, however, the risk of mass air-assaults is decreasing, ac- cording to Hutchings and Street [86]. As such, more value should be obtained by shifting developmental efforts towards countering high-technology, jamming, stealth and long-range stand-off WSs.
Extent of WA switching
Drastic changes in threat values, as was identified in Chapter 5, may have a detrimental effect on the functioning of the WA subsystem. These threat value variations are undesir- able from a C2 decision-making perspective, since these threat value variations will affect the suggestions provided by the WA subsystem. This detrimental effect may materialise as a switching of WA (WS-to-threat assignments) recommendations. This switching may become a problem if the WA recommendations changes rapidly and excessively over a small subset of consecutive time-stages [118]. This kind of behaviour may be ascribed to the changing threat values as well as the various WA stochastic elements (SSHP values and heuristic solution methodologies).
Consequently, this switching may cause confusion on the part of the FCO and thereby result in the FCO questioning the credibility of the TEWA system’s results. This, in turn, may lead to disagreement uncertainty, as descried in §7.3.3. This confusion may, subsequently, result in the operator relying on his own judgement by resorting to heuristic
rules (the reader is referred to§7.3.2 for a more in-depth discussion) instead of relying on the TEWA DSS, thereby resulting in sub-optimal responses.
In order to mitigate this problem, the use of threshold values was suggested in §6.4.1. A new WA solution should only be accepted if the new solution is deemed significantly improving (i.e. if the new objective function value exceeds a specified threshold tolerance over and above the current objective function value). Allouche [9] suggested the use of smoothing techniques — specifically Kohonen’s self-organizing maps8 — to smooth the trajectory of a threat in an attempt to render the threat values more gradually changing. Allouche’s focus was on missiles in a sea-based environment, but these methods may be tailored and applied to a GBAD environment.
Required computational resources
It is important to ensure that the FCO has enough time to utilise the results generated by the WA subsystem. Also, depending on the environment in which the TEWA system is implemented, there might be memory restrictions (e.g. micro-controller architectures or FPGA restrictions). There is often a trade-off between the time complexity and memory complexity of an internal TEWA algorithm — increased memory consumption generally corresponds to faster execution times, and vice versa. The increased need of advanced visualisation capabilities (see§7.4.2) also present further challenges to the system designers in terms of being able to ensure that information is updated continuously and visualisations are rendered quickly, so as to assist the operators with their decision-making cycle. Although Matlab is used in this thesis — which is a high-level programming language that is known to perform slower when performing certain operations — the execution times of a Matlab simulation may serve as an initial estimate by which to understand the scope of required computational resources. In practice, however, when the system is implemented using a lower-level programming language (C or assembly), the execution time may decrease by several orders of magnitude.
8.7 Chapter Summary
This chapter opened in §8.1 with an introduction to the evaluation of TEWA systems, after which the underlying concepts of an SoS approach was explained in §8.2. After understanding the notion of an SoS, a possible methodology to adopting an SoS approach was elucidated in§8.3. This approach, essentially, requires the adoption of a scenario-dependent simulation model in an SoS analysis context, as opposed to an analytical approach involving unrealistic hypothetical scenarios. Because the outputs of a TEWA system strongly depend on the specific GBAD setup as well as the working TEWA algorithms, such a systems approach to performance evaluation is expected to provide more significant results.
After a possible approach to performance evaluation had been elucidated, three possible scenario- dependent performance evaluation approaches were described and qualitatively evaluated in §8.4. These scenario-dependent approaches include prototype evaluation with end-users, single scenario evaluation and batch-simulations. Four TEWA-specific performance evaluation metrics were also proposed in §8.5. Each metric was designed so as to provide different insights into the results generated by a TEWA simulation. The metrics include a survivability metric, an economy metric, an engagement effectiveness metric and an adaptability metric. This chapter closed in §8.6 by clarifying three practical considerations that should be borne in mind when using a TEWA performance evaluation simulation framework.
CHAPTER 9
Worked Example
Anomalies are things that either do not happen and should, or that do happen and should not.
— Herman Kahn
Contents
9.1 Experimental Approach . . . 150 9.2 GBAD Scenario Deployment . . . 150 9.2.1 Defended Asset Placement . . . 152 9.2.2 Weapon System Placement . . . 152 9.2.3 Threats Attack Profiles . . . 152 9.3 Threat Evaluation Application . . . 154 9.4 Weapon Assignment Application . . . 157 9.5 Performance Metrics Calculation . . . 162 9.6 Chapter Summary . . . 164
The purpose of this chapter is to demonstrate the workability of the concepts, strategies and algorithmic models reviewed and presented in this thesis. In addition, the demonstration of the constituent TE and WA models — applied to a new GBAD scenario — also serves as an additional verification step to ascertain the correct functioning of the algorithmic models. This demonstration is achieved by means of a comprehensive, near-realistic, but hypothetical scenario created by Potgieter [155] in cooperation with a military expert, Visser [219].
The chapter opens with an overview of the scenario setup considered, and this is followed by a description of the attack profiles of the attacking aircraft and the positioning of WSs in the scenario. The physical elements within the simulation are modelled as described in Chapter 4. After having gained an understanding of the functioning elements within the simulation environ- ment, the outputs of the TE subsystem, as explained in Chapter 5, is provided and interpreted. This TE information is, in turn, used to generate an allocation suggestion of WSs to threats, as described in Chapter 6. The chapter closes with the calculation of the system performance evaluation metrics presented in Chapter 8.
9.1 Experimental Approach
As explained in §8.3, a constructive discrete-event simulation framework is more suitable for the performance evaluation of a complex TEWA DSS in its totality than a purely reductionistic (analytical) approach where TE and WA are tested in isolation. Such a discrete-event TEWA simulation has strong parallels with the notion of wargaming [104]. The steps in a simulation study, as detailed in §4.1.2, are therefore similar to the required steps of a wargaming exercise. These steps are paraphrased below:
1. Define the problem and objectives of the experiment. The purpose of this simulation is to demonstrate the workability of the concepts, strategies and algorithmic models reviewed in the previous chapters of this thesis.
2. Prepare input data. In the case of this experiment, the inputs include details on the GBAD scenario setup — positions of DAs, WSs as well as the input coordinates of the threats that are used for the flight path generation. The preparation of the inputs also includes detailing the properties of all the simulation model entities. The modelling approaches described in §4.2.1 are used for the representation of the simulation model entities (e.g. DAs, sensors, WSs and threats).
3. Execute a preliminary experiment and execute production runs. The preliminary exper- iment includes a performance appraisal of the TEWA system described in the previous chapters in the context of the illustrative example introduced in §4.6. Part of the pre- liminary experiment entailed a detailed analysis of the TE and WA models implemented so as to validate and verify their functioning. Different limitations of the WA and TE subsystems were also identified throughout the thesis and mitigation strategies were im- plemented. The final production runs are used for the performance evaluation of the system in this chapter.
4. Analyse simulation outputs and system performance measures. After completion of the production runs, it is possible to commence with the performance evaluation of the con- stituent algorithmic model combinations. As explained in §8.3, the outcome of a TEWA simulation performance evaluation study can only be certain possibilities associated with certain events (for example, the set of possible TEWA-cycles during which a threat was successfully engaged). From this evaluation, areas for further improvement may be iden- tified.