While client-side tracking enables the creation of rich analytics, it is also possible to capture analytics data using an infrastructure-based approach. Similar to client-side tracking, the overall aim of infrastructure-based tracking in the context of digital signage is to capture contextual information about viewer navigation patterns and to support display personalisation. This is an alternative approach to collecting viewer mobility data as we proposed as part of Figure3.1. In this section we describe the most commonly used approach of infrastructure- based tracking in the form of Wi-Fi fingerprinting – a commonly used technique across commercial environments such as shopping malls. The advantage of infrastructure-based tracking is that viewers are not required to install a dedicated application on their mobile phones, and the environment is not required to be equipped with additional tracking technology. Typically, data collected in the environment is used to understand common walk paths and navigation patters inside the enclosed environment [Kha+13;Jay+16].
We have designed an infrastructure-based display personalisation system in the context of LiveLabs, “a first-of-its-kind testbed that is deployed across a university campus, convention centre, and resort island and collects real-time attributes such as location [...] from hundreds of opt-in participants” [Jay+16]. In particular, LiveLabs has been deployed at a large convention centre located in Singapore, collecting location information of all visitors present in the space with a frequency of approximately 15 seconds. LiveLabs consists of an event-based API that allows external services to access current location information at the point at which they have been captured.
3.4.2.1 System Design and Architecture
The system architecture consists of six modular components that have been based on Tacita (Section3.4.1): context data fetcher, pattern recognition, content creation and selection,
infrastructure connectors, context store, and areal-time and event-based content trigger
engine.
Similar to Tacita, the system relies on proximity information of viewers to public display and the specification of trigger zones around individual displays which are used to detect viewer enter and exit events to change the content shown accordingly. Due to the availability of different mechanisms that can be used to detect viewer proximity, we included theContext
Data Fetchercomponent that can be used to plug in any external source for location tracking
including Wi-Fi fingerprinting provided by LiveLabs [Jay+16]. Each external data source is modelled as an individual process with access to a local context store through which location update events are communicated to other system components in real time. Further, external data sources are required to provide a unique user identifier and the users’ location points as a basic set of information. The proximity detection of viewers to displays is computed within thereal-time content trigger module: each location point of a potential viewer is compared with the set of trigger zones of displays to detect whether individual viewers are within a display trigger zone. The system utilises an event-based approach: real-time
3.4 Capturing Viewer Mobility Data 74
Personalisation System Architecture Context Data Fetcher
Pattern Recognition
Content Creation and Selection
Infrastructure Connector
Location Traces Import
Import from other data sources Web-based GUI Allocation of content to interests Duration-based Pattern Recognition other...
API / Export module of content for Suntec API / Export module of
content for e-Campus Third-party data sources
LiveLabs location tracking, Suntec displays log, additional sensor data such as video analytics, escalator
sensors, etc.
Providing content Allowing content creators to submit
content to the system and tag the content to specific user interests.
Digital Signage Network
The infrastructure connector exports the data so that it can be consumed by the digital signage network - including the allocation of content to specific displays and
perhaps also the export of individual content items.
Real-time Content Trigger
Location prediction based on last seen Content trigger based
on location
Database System
Real-time location trace and Prediction Historic dataset of location traces
Figure 3.10:System architecture for infrastructure-based tracking and public display personalisation integrating LiveLabs [Jay+16] real-time location tracking.
3.4 Capturing Viewer Mobility Data 75
location updates are passed on to the real-time content trigger component that detects whether viewers are in proximity to displays and finds corresponding rules for content selection for individual viewers. Content preferences are computed on a constant basis within thepattern
recognitioncomponent that, based on historic navigation patterns stored in the data store,
determines potential viewer interests—for example, based on duration spent in certain rooms and locations. The actual content and content selection rules are manually supplied by the space owner and display providers and defined within thecontent creation and selection
module. Rules, for example, can consist of location and time dimensions. Theinfrastructure
connectorcomponent is, similar to thedisplay gatewayin Tacita, responsible for the mapping
of visitor proximity to the actual display and immediate content change requests on the display. More details on the selection of content based on historical traces of viewers is described in Chapter5(Automated Use of Pervasive Display Analytics).
3.4.2.2 Opportunities for Data Collection
In the context of infrastructure-based viewer tracking, collected datasets exclusively originate from the space owner and can be categorised asserver-side no sharingin accordance with our data categorisation framework described in Section3.3. Whilst advantages of this approach include the absence of required viewer opt-ins and dedicated mobile phone applications, the quality and accuracy of using infrastructure-based sensing may appear to be of lower quality depending on the technology used.
However, Wi-Fi fingerprinting enables the collection of comprehensive navigation patterns of viewers within a space beyond the vicinity of the display and provide us with “a broader view of the surroundings” [Section3.2.2, Space Owners] beyond the immediate vicinity of displays. More specifically, the collected datasets include anonymised user identifiers, location points within the space, date- and timestamps, and an accuracy measure. The rich dataset allows us to compute viewer proximity to displays, dwell times at any location within the space, and map user locations to specific conference venues and events. The resulting dataset gives us insights into potential interests of individual viewers: for example, a long dwell time in a certain conference room in the convention centre can be mapped onto a specific event and used later-on for personalised content selection. In addition to viewer-related data, we also capture content displayed on each digital sign within the venue that has been integrated in to our system. The resulting dataset of viewer mobility paths can be enriched by accessing Pheme to retrieve content shown and timestamps of associated displays.
3.4.2.3 Considerations
Whilst the collection of real-time location data is fundamental to this system, the use of the collected information for the benefit of the visitor is important regarding the acceptance of the system—in contrast to systems that collect mobility data or audience demographics purely for the analytics and the benefit of the space or display owner.
3.4 Capturing Viewer Mobility Data 76
Using location traces to improve viewer experience requires the addition of a dimension that allows the association of content with displaysand date and time attributes to show content in the future according to viewer presence in the past. To better understand which display content works best for viewers and improve content selection algorithms, the lack of a feedback loop (i.e. capturing viewer responses to content) provides an additional challenge in the design and development of potential machine learning algorithms and understanding which content worked best and was most effective. However, real-time location traces can be a suitable mechanism to identify groups of users to target and, in particular, automatically generate targeted content based on current contextual information from the vicinity of the display without using potentially more privacy invasive technologies such as face recognition or face identification.
3.4.2.4 Implementation
The system was integrated with LiveLabs [Jay+16] and the pervasive display deployment at Suntec Singapore [LG 13] consisting of a large high-definition video wall and over 100 digital signs located across the conference venue. The back-end components have been implemented entirely in Python whilst the front-end components and content player were written using Web technologies and AngularJS. The display network at Suntec is using the Four Winds digital signage platform4. Due to the limited functionality in the version deployed at Suntec, we implemented a dedicated Web-based public display client for dynamically displaying content that has been shown in full screen through the Four Winds display player. The client page and the back-end communicate through Web sockets, supporting real-time communication to immediately change the content based on viewer proximity.
3.4.2.5 Evaluation
We note that the evaluation of this system is described in Chapter6(Trials), Section6.4.