Major topics of current research in HLIF are presented by Blasch et al. in [34]. In their survey paper, the authors extract the top ten trends of HLIF from the conference pa- pers and panel discussions published within years 2000 to 2011, and categorize them into five main groups of Data and knowledge representation, Situation, threat, and impact assessment, Systems design, Evaluation, and Information management. Furthermore, Un- certainty Analysis and Semantics and Ontologies trends are featured as the most important areas of study that come before other crucial trends such as Reference Model Definition, Social Behavioral Model, and Resource Planning. Therefore, the literature review in this section is narrowed to the analysis of some well-defined HLIF frameworks, and to the sub- stantial research work addressing uncertainty management, and semantics and ontology representation in HLIF.
General HLIF Frameworks
A comprehensive schema with seven building blocks for designing an HLIF system is in- troduced by D.A. Lambert in [124]. In this schema, the first building block discloses a three level assessment paradigm, and presents a de-constructed JDL model [226] in which the 4th level of its revised version [208] is implicitly embedded as an assessment unit into levels 1 to 3. The resulting model consists of three major levels aiming to render object, recognize relationships, and assess the impact of those relationships, respectively. In the second building block, the fundamental steps in adding the machine-readability capability to situation and impact assessment is discussed and major steps through which syntactic tokens acquire meanings using a sequential schema, are presented. Moreover, the third building block aims to represent the human mental status in machines by defining beliefs, expectations, and anticipations, which are generally called awareness. The State Tran- sition Data Fusion (STDF) model, is the content of the fourth building block that goes through the DF processes from the lower level of object assessment to higher levels of situation and impact assessment. Fifth building block demonstrates the applicability of social behavior models, wherein the distributed methods such as Distributed Data Fusion (DDF) techniques, and ubiquitous fusion are among the main topics. In DDF, each level of JDL model will perform its own task in a distributed approach, and then the local information is merged to form global information about the current state. Furthermore, ubiquitous fusion allows the objective of a particular agent to enter a society (i.e., VANET environment), for further examination upon acquiring the acceptance of all other agents, or at least a significant portion of them. Therefore, the agents participate in a society, and
increase the robustness of their DF paradigm. Finally, sixth and seventh building blocks evaluate different methods in demonstrating the results to a user-level endpoint, and in studying the human condition in different aspects of this demonstration, respectively.
In order to have a generic HLIF model, active role of human, and a bi-directional inter- action between human and technology should be taken into account. Nilsson et al. in [166] propose this idea by first arguing the limitations of traditional fusion models, specifically JDL model [208] and OODA loop [135], in incorporating human decision making models, and using humans inherent cognitive capability to enrich the fusion processes. Therefore, Nilsson et al. propose their human-technology driven environment in which users are the active parts of the fusion process and while receive useful knowledge from the provided technology through artefacts, help to improve the awareness of the current situation by utilizing their decision making capability, and removing technologically related flaws such as untrustworthy results. The first steps towards a human-technology interactive fusion environment is taken by employing the distributed cognition concept that is originally introduced by Hutchins in [104]. Distributed cognition studies the understanding of the dynamic flow of information (i.e., a process) through a systematic organization of compo- nents (i.e., either humans or technological artefacts). Technically speaking, these processes are deemed distributed in three ways: across the components, between internal and exter- nal aspects of components, and over time.
Semantics and Ontology Representation
Based on the concept of Situation Theory [21,20,19], Kokar et al. [119] define a situation awareness framework based on an ontology described using the OWL, which they refer to as the Situation Theory Ontology (STO). The STO creates different ontology-based concepts of the situation theory by defining OWL classes and connections for objects, types, and their relationships. Furthermore, a situation can be easily represented using a set of classes related to that situation, with appropriate relationship definitions among them. Lastly, new situations are inferred by creating a knowledge base containing horn clauses. Although it is a well-organized framework for situation semantics representation, STO lacks uncertainty management, which is deemed an important aspect of a HLIF model.
Mapping ontologies is another important application of using ontologies in HLIF sys- tems which is also discussed by Wache et al. in [220] in two cases. In the first case, an ontology can be mapped to an information source, with the major steps of Structure Re- semblance, Term Definition, Structure Enrichment, and Meta Annotation. In the second case, two different ontologies can be mapped to each other, i.e., inter-ontology mapping, through employing pre-defined mappings, lexical and semantic relations correspondence,
and top-level grounding. At the end, Wache et al. propose three steps for creating an ontology and having an engineering vision on its development process. The first step iden- tifies the underlying scope being processed. Moreover, building the ontology by capturing and coding it, and integrating different ontologies, are the contents of the second step, and finally, the third step evaluates the constructed ontology accordingly.
Abstract semantics and ontology representation is discussed by Little and Rogova in [137] who address the problem of defining formal structures for different entities, their attributes and properties, and relationships between objects, etc.. This is done by using hybrid approaches and formal ontologies. In their proposed method, Little and Rogova aim to make the in-between connections between a higher-level ontology with more abstract structure, and a pre-determined domain-specific ontology.
Heintz and Dragisic in [96] tackle semantics representation by using the idea of anno- tating sensors based on their semantic structure, and reasoning by semantic information integration. Therefore, a source is generating relevant data for a service if their pre-defined ontologies match. Furthermore, they propose an application independent framework which can be customized to find a set of information sources satisfying the given demand for a specific service.
In fact, lacking uncertainty management capability is a common drawback in most of the frameworks that only care for representing the ontological and semantic relationships between the entities existing in a specific environment. Therefore, it is necessary to develop situation awareness frameworks which are also capable of modeling uncertainty.
Uncertainty Management
The fundamentals of a dependable and generic HLIF method for handling uncertainty is proposed by Karlsson in [112]. In his technical report, Karlsson categorizes the meth- ods dealing with uncertainty, so-called Uncertainty Management Methods (UMMs), into three groups of Bayesian, Dempster-Shafer, and Imprecise probability approaches. This technical report raises some interesting open questions such as the possibility of fusing temporal attributes, i.e., information in present, past and future, as well as the definition of evaluation metrics for measuring the performance of different HLIF systems.
Costa et al. in [55] propose the Uncertainty Representation and Reasoning Evaluation Framework (URREF) to improve the system-level metrics such as timeliness, accuracy, and confidence. In other words, their main goal is to study the effect of uncertainty on IF systems. Therefore, they present an abstract model in which different uncertainty handling tools such as probabilistic methods, Dempster-Shafer theory, and Fuzzy Sets, can
be used in a plug-and-play fashion. Furthermore, they define an ontology for the proposed framework to make sure that all the evaluations are semantically sound. Different types of entities, and relationships between different objects in the domain of uncertainty handling are determined through this ontology.
An Object-oriented Probabilistic Relational Model (OPRM) is introduced by Howard and Stumptner in [101]. This model, which is a new language for First-order Probabilistic Logic (FOPL), aims to handle situation assessment by formalizing object and relationship recognition, IF at different abstract levels, and handling uncertainty and temporal nature. The main structure of OPRM consists of a set of classes featured with a set of descriptive and reference attributes. OPRM is also equipped with a probabilistic component which defines probability distributions on attributes to model uncertainty on them. One of the most strong capabilities of OPRM, is its uncertainty handling power which is imposed on existence, attribute, and structural uncertainties.
Uncertainty handling is tackled in a situation awareness framework for transportation, proposed by R¨oeckl in [185], which uses dynamic probabilistic causal decision networks to help making decisions with maximum utility. Both forward and backward propagation are studied to demonstrate the capability of the framework in making decisions based on the observed evidence, and in optimizing evidence selection for a given decision. However, this method cannot be used in a wide range of applications due to the lack of semantics representation capability. Besides, when the complexity of the environment increases, it is unlikely to demonstrate efficient inference time, which is inevitably inherited from BN.