Some datasets, including van Kasteren’s House A dataset (vKNEK08), which we use later as part of the evaluation, include the notion of idleness: periods of time over which any non-specific situation occurs. The literature describes approaches to situation recognition that both include and ignore idleness, and we later evaluate this work taking both into consideration.
In both a standalone classifier and in the genetic ensemble, classification ofidlesitu- ations can only be achieved by treating idle time slices as a regular situation. However, given that idleness does not describe a fixed phenomenon, but the set of all activities not otherwise specified, generation of a concise and intelligible specification set or decision tree to represent idleness can be difficult.
The semantic hierarchy approach more easily accommodates the notion of idleness, through incorporating its detection within various classifiers in the hierarchy as a filter-like construct. We have designed a two-fold approach to achieve this. Firstly, all singleton specifications are re-designated as a paired grouping consisting of the singleton and theidlesituation:
Situation’=Situation∪Idle
Secondly, classifiers are generated at each of the leaf nodes to support differentiation of the singleton from theidlesituation. This process can be used to redefine the standalone classifier as a two-level classification task, or can be applied to the semantic hierarchy already constructed. The latter approach is depicted in Figure 5.3.
Hierarchical Ensemble Classifier (+ Idleness)
Cubicle'
Working' Reading'
Root
Break' Cubicle' Café' Meeting'
Café' Coffee' Lunch' Coffee' Idle Coffee Lunch' Idle Lunch Working' Idle Working Reading' Idle Reading Break' Idle Break Meeting' Idle Meeting
Figure 5.3: An illustration of the semantic hierarchy ensemble, extended to handle idle time slices.
This approach is based on the assumption that it is easier to generate multiple classifiers capable of profiling idleness when compared to singleton situations, than to have a single model that attempts to differentiate idleness from the set of all situations. Further to this, it allows the higher layers of the classifier to be trained only on situation specific data, and offers the potential to customise how theidlesituation is disambiguated at the level of each situation.
5.4
Summary
This chapter discussed approaches to constructing ensembles with the goal of obtaining more accurate and reliable decisions than a standalone model and, in the case of the semantic hierarchy, to simplify the recognition process through modularising the structure of the recognition task.
We began with a primer on the use of Dempster-Shafer theory, a mathematical theory of evidence for combining information from multiple sources, that is popular in the design of pervasive- and sensor-systems. Its ability to preserve uncertainty and the possibility of assigning confidence to sets of situations, not only situations individually, plays a later role in consensus finding.
Next, we outlined two different approaches to constructing ensembles: the genetic ensemble and the semantic hierarchy. For each we overviewed the underlying concepts, design and implementation methodology, and gave an example illustrating their use. The genetic ensemble is based on manipulating the fitness function of the genetic algorithm during the classifier generation process. It involves the generation of a set of classifiers, one for each situation, with the overall result computed by combining the output of each.
The semantic hierarchy exploits expert knowledge about how situations are related. It involves the generation of multiple classifiers to successively refine a classification from abstract groupings to specific situations. Results are computed by following an execution path thorough multiple set of classifiers, where the output of one situation set governs the selection of the next classifier to evaluate. A further contribution explores how the semantic hierarchy is extended to support detection of ‘idleness’—periods of time over which non-specific situations occur.
In the next chapter we prepare datasets for the evaluation of the techniques developed in this and earlier chapters, through mapping them to our information space model.
CHAPTER
SIX
DATASET PREPARATION
To this point we have introduced a reusable top-level ontology model that provides a common substrate for developing domain and application ontologies for pervasive environments, two hybrid situation recognition models based on the constructs of this model, and approaches to ensemble classification that aim to provide improved situation recognition accuracy and intelligibility. Before evaluating these algorithms in Chapter 7, this chapter describes the process of readying a dataset for application of these techniques. We refer back to 3.8 for more general discussion surrounding the engineering effort this process requires.
The evaluation requires sensor datasets that are annotated with ground truth (situations asserted to be occurring at the time sensor readings are recorded) and that contain complex enough situation and sensor arrangements so as to support the building of concept hierarchies.
To these ends we have selected five datasets and grouped them into two categories:
• Smart-office
– TheCASLdataset (MYCD09a) captures the workday situations and sensor traces associated with a doctoral student in a laboratory setting.
– TheInk-12dataset captures the workday situations and sensor traces of a doctoral student in a home-office setting; collected for this research as a study modelled on the CASL dataset.
• Smart-home
– Thevan Kasterendatasets (House A, B and C) (vKNEK08; vKEK10a): Three datasets that capture the sensor traces and typical daily activities in three single occupancy houses.
While the datasets we have selected describe smart-environments, they have been chosen primarily because they are marked with ground truth, and recur in the literature, supporting comparative evaluation of the techniques we have developed. We believe the same set of techniques described in this thesis can be applied to other sensing domains, such as wearables, where a representative model of the information space can be constructed.
We overview the standard methodology for preparing a dataset in Section 6.1, be- fore applying this methodology to each of the smart-office and smart home datasets in Section 6.2 and Section 6.3 respectively. A summary is presented in Section 6.4.
6.1
Methodology
To be useful, a situation recognition dataset must provide, as a minimum requirement, two features: timestamped traces from sensors deployed in an environment, and a ground truth that describes the occurrence and duration of situations that occur therein. Both are necessary in order to construct a snapshot of the state of the sensors at any given time, and associate it with the occurrences of situation in preparation for the application of learning (or in the case of unsupervised techniques where labelling of situations is not required, support their evaluation).
Collecting accurate ground truth annotations can be a challenging and expensive task, and is prone to error. In the CASL, Ink-12, and House C datasets, subjects recorded their activities in handwritten diaries, while in the House A and House B datasets subjects provided annotations via bluetooth headsets in concert with speech-to-text recognition. Both approaches risk subjects forgetting to annotate activities, while handwritten diary timestamps are, naturally, less accurate than their headset-captured equivalents. Alternative techniques may yield better results; one such possibility is camera tracking, however the accuracy gain comes as a trade-off with the cost to purchase and install the equipment and the typical need to manually transcribe the footage to annotations (TIL04; CYW05). In all cases, the Hawthorne or observer effect, whereby subjects may act differently given the knowledge that they are being ‘watched’, may affect how accurately the collected data reflects the subject’s activities
in an unmonitored situation (MWI+07).
Given an annotated dataset, four processes must be carried out to prepare it for use with the situation recognition techniques described in this thesis: situation identification, situation model construction,context model construction, andsensor data to context statement mapping. These tasks require varying levels of ‘expert’ knowledge which
may come from various sources including sensor manufacturers, sensor installation engineers, and the observed subjects. We outline each process below.
Situation Identification Here, the task is to enumerate the set of situations that the recognition technique must differentiate. This may be defined before data collection begins, or may be derived from a ground truth collected in free form. Knowledge of the target environment should be used to restrict the set of situations to those deemed likely determinable from the environment’s sensing capability. For example, intuitively the situationwatch_tvshould only be included in the enumeration of candidate situations if at least some of its associated characteristics (for example, the television being switched on, or the subject being sat in a particular seat) can be directly sensed or inferred.
Situation Model Construction After the set of situations thought to be discernible has been identified, the next step is to explore if any of the relations defined in Sec- tion 3.3 can be used to connect them, either directly, or through the construction of an intermediate concept. For example, the identified concepts ofin_carandin_busmight be united under the encompassing concept ofin_vehicle. These connections play a role in structuring any hierarchical ensemble.
Context Domain Model Construction Next, and again following the model described in Section 3.3, concept hierarchies are constructed for each information domain for which sensors provide information. For example, Figure 6.1 illustrates one possible encoding of time, with the finest grained concepts representing hours of the day (H0-H23), and the more abstract terms representing labelled periods of the day first in 3, and then 6 hour durations. For clarity, the adjacency relations between the start and end of the day are not depicted. Depending on the situations to be recognised, other temporal encodings may be more useful, for example, a division based on the concepts before_office_hours,office_hours, andafter_office_hours.
Sensor Data to Context Statement Mapping Finally, the raw sensor data is mapped to context statements using the defined concept hierarchies. This occurs in three (potentially overlapping) steps: In the first step, all sensor data corresponding to a particular domain is resolved to a single value. In the second step, a mapping function projects the result of the first step onto the concept model. In the third step the corresponding context statement is constructed.
The resolution of multiple observations to a single value may be achieved in several ways. In the simplest case, all observers agree on the value and therefore the resolution
is trivial. If the reported values differ, it may be the case that the mapping function evaluates to the same concept in each case, again resolving the conflict. If this is not the case, a number of strategies may be adopted: selecting the most recent data, averaging reported values (if numeric), or selecting the majority value among many.
The mapping function may be realised as a one-to-one relation, whereby each possible sensor value is mapped to a single concept in its associated domain model, or as a one-to-many allocation of confidences across a set of concepts; for example, mapping the time ‘13:24pm’ to the concept H13 with confidence 1, or the temperature ‘18○ Celsius’ to the conceptsColdandHotwith confidences 0.7 and 0.3 respectively. Finally, the context statement is constructed from the resolved values. For example,
[:system,:hasTime,:H13], or[:kitchen,:hasTemperature,:Hot].