UNIVERSIDAD POLITÉCNICA DE MADRID
ETSI y Sistemas de Telecomunicación y ETSI y Sistemas Informáticos
Master Universitario en Internet of Things
MASTER THESIS
DEVELOPING SCALABLE IOT SERVICES BASED ON THE LPWAN TECHNOLOGY ECOSYSTEM: AN OPEN
SOURCE APPROACH
MIGUEL DEL CASTILLO CAMPOS
JULY 2019
Trabajo Fin de Máster
Máster Universitario en Internet of Things
Director /Cotutor
Firma:
Firma:
Firma:
Tribunal
Developing scalable IoT services based on the LPWAN technology ecosystem: an open source approach
Miguel del Castillo Campos
Rubén de Diego Martínez
Iván Corredor Pérez
Eduardo Barrera López de Turiso
Antonio da Silva Fariña
Fernando Pescador del Oso
Agradecimientos
El año que va de Julio de 2018 a Julio de 2019 ha sido, sin lugar a duda, el año más complicado de toda mi vida. Es por ello que quiero aprovechar la oportunidad que me da el estar presentando esta memoria del TFM para poder agradecer a todos aquellos que me han ayudado de una manera o de otra a sobrevivir a este año.
A mis compañeros del máster, en especial a José Luís, Ramón y el Crack. No estaría escribiendo esto si no hubiera tenido vuestra ayuda y vuestro apoyo casi desde el momento en que nos conocimos.
A aquellos que se embarcaron conmigo en una aventura que fue corta y, sobretodo, intensa y que han sabido entender la importancia de la familia por encima de los negocios.
A Iván y Rubén por haber echado horas de más y haberme ayudado en todo lo que han podido y más para llevar a buen puerto este TFM.
A los “paveros” que una vez más han estado apoyando desde la distancia en cada ocasión que han tenido.
A mis suegros, ahora sí puedo decirlo, que mostraron su apoyo y comprensión cuando más lo necesitaba.
A mis padres, a mi hermana y en especial a la canija que ha conseguido hacerme sonreír cuando no conseguía ver motivos para hacerlo.
A mi cuñado preferido, que ha demostrado ser una persona excepcional y el mejor cuñado del mundo, que me hace sentir que tengo el hermano mayor que nunca tuve.
Y, por último y por ello más importante, a mi mujer, quien ha sufrido día a día conmigo en este año y que ha tenido la mayor paciencia del mundo para aguantarme y ayudarme en todo lo posible. Entre ella y la peluda, han conseguido que no pierda la cabeza.
Gracias.
“Nunca llegarás a destino
si te dedicas a tirar piedras
a cada perro que te ladre”
Winston Churchill
i
Index
Index ... i
Figures Index ... iv
Tables Index ... vii
Summary ... 1
Resumen ... 3
1 Introduction ... 5
1.1. Thesis Outline... 8
2 State of the art... 9
3 Description ... 15
3.1. System Requirements Specification ... 17
3.2. System Design ... 18
3.2.1. Architecture Overview ... 18
3.2.2. Rosetta Architecture ... 19
3.2.3. Management Tool Architecture ... 20
3.2.4. Functionalities Sequence Diagrams ... 21
3.2.4.1. Data Upload ... 21
3.2.4.2. Component Management ... 22
3.2.4.3. Rosetta Configuration ... 23
3.2.4.4. Commands sending ... 23
3.2.5. Databases Definition ... 24
3.2.5.1. Rosetta Gateway Database ... 24
ii
3.2.5.2. Management Tool Database ... 26
3.2.6. Information Model ... 26
3.3. Implementation Design ... 29
3.3.1. Functional Diagrams ... 32
3.3.1.1. MessageRouter ... 32
3.3.1.2. Serial Communications Manager ... 33
3.3.1.3. Drivers ... 33
3.3.2. Class Diagrams ... 34
3.3.3. Development Resources ... 40
3.3.3.1. Interface Development Environments ... 40
3.3.3.2. Programming Languages ... 41
3.3.3.3. Hardware Devices ... 41
3.3.3.4. LPWAN Protocols ... 41
3.3.3.5. Databases ... 42
3.3.4. Implementation Details ... 42
3.3.4.1. Publish-Subscribe-based message protocol ... 42
3.3.4.2. Databases ... 43
3.3.4.3. Interfaces ... 43
3.3.4.4. Semi Plug and Play ... 44
3.3.4.5. Management Tool ... 44
4 Experiments and Use Cases ... 47
4.1. Experiments ... 49
4.1.1. Semi Plug and Play Test ... 50
iii
4.1.2. Driver Installation ... 50
4.1.3. Data Upload Using Ethernet ... 52
4.1.4. Data Stored in Database Representation ... 52
4.1.5. Data Uploading Using NB-IoT ... 53
4.1.6. Stress Test ... 53
4.1.7. Continuous data upload test ... 54
4.2. Use Cases ... 56
5 Budget ... 57
6 Conclusions and Future Work ... 61
6.1. Conclusions ... 63
6.2. Future Work ... 63
7 References ... 65
iv
Figures Index
Figure 1. Generic Gateway Architecture 12
Figure 2. Architecture Overview 18
Figure 3. Rosetta Architecture 19
Figure 4. Management Tool Architecture 20
Figure 5. Data Upload Rosetta Gateway Point of View 21
Figure 6. Data Upload Management Tool Point of View 21 Figure 7. Component Installation/Update Management Tool Part 22
Figure 8. Component Installation/Update Rosetta Gateway Part 23
Figure 9. System Start Configuration. 23
Figure 10. Command sending 24
Figure 11. Gateway Database 26
Figure 12. JSON Rosetta Message 27
Figure 13. Command Message Format for Components Management 28 Figure 14. Command Message Format for Serial Configuration Update 29
Figure 15. OSGi Bundle Lifecycle 30
Figure 16. Rosetta Gateway Implementation 31
Figure 17. Management Tool Implementation 32
Figure 18. Functional Diagram MessageRouter Bundle 32 Figure 19. Functional Diagram CommManager Bundle 33
Figure 20. Generic Driver Bundle 33
Figure 21. Internet Raspberry Pi Driver 34
v Figure 22. DatabaseManager and MqttRosettaClient 34
Figure 23. MessageRouter 35
Figure 24. SerialConfig 36
Figure 25. SerialComm 37
Figure 26. CommManager Bundle 38
Figure 27. Driver 39
Figure 28. MessageRouter 40
Figure 29. System Topics 43
Figure 30. Commands Screen 44
Figure 31. Data Display Screen 45
Figure 32. Gateway Status List 45
Figure 33. Proof-of-Concept 49
Figure 34. Plug and Play Test 50
Figure 35. Default Bundles 50
Figure 36. Waspmote Driver Installation 51
Figure 37. Current Installed Bundles 51
Figure 38. Waspmote Driver Stopped 52
Figure 39. Dynamic Table and Charts 52
Figure 40. Static Table and Charts 53
Figure 41. Stress Test 54
Figure 42. Continuous test beginning 54
Figure 43. Continuous Test ending 55
Figure 44. Use Case 1 56
vi
Figure 45. Use Case 2 56
vii
Tables Index
Table. I. Requirements Compliance 29
Table. II. Proof-of-Concept Components 49
Table. III. Proof-of-Concept Budget 59
1
Summary
The aim of this Master Thesis is to develop a proposal using open source resources for building a suitable Gateway which could be remotely managed, as well as open hardware characterizing the IoT infrastructure, i.e. the Gateways and the sensor devices.
The management of the Gateways includes the deployment of functionalities in a remote way [1] [2], allowing changing the behaviour and configuration of the gateway and the underlying WSN.
Moreover, the Gateway is able to seamlessly integrate data streams generated from WSNs into the cloud service ecosystem independently of the communication protocol applied [3].
A Proof-of-Concept has been developed in order to demonstrate this characteristic in a real use case.
This proof-of-concept has been constructed using Eclipse Kura [4] installed on a Raspberry Pi 3B [5] as the gateway core and using as interfaces a Waspmote [6] for the data collection simulation and a FiPy [7] to implement the data upload (uplink). Also a version of the system with uplink and downlink using the Ethernet of the Raspberry Pi 3b has been developed.
2
3
Resumen
El propósito de este Trabajo Fin de Máster es desarrollar una propuesta, utilizando recursos abiertos, para construir un Gateway que pueda ser gestionado de forma remota, así como hardware abierto que caracterice la infraestructura IoT, es decir, Gateways y dispositivos IoT.
La gestión de los Gateways incluye el despliegue de funcionalidades de forma remota [1] [2], lo que permite modificar el comportamiento y la configuración del Gateway y la red de sensores inalámbricos subyacente (WSN por sus siglas en inglés).
Además, el Gateway integra a la perfección los flujos de datos generados desde las WSN en los servicios de la nube independientemente del protocolo de comunicación utilizado [3].
Se ha desarrollado una prueba de concepto para demostrar esta funcionalidad en un caso real.
Esta prueba de concepto se ha construido utilizando Kura [4] instalado en una Raspberry Pi 3B [5] como núcleo del Gateway y utilizando como interfaces una Waspmote [6] para la simulación de la recolección de datos y una FiPy [7] para implementar la carga de datos (enlace ascendente). También se ha desarrollado una versión del sistema con enlaces ascendente y descendente que utiliza la conexión Ethernet de la Raspberry Pi 3B.
4
1 Introduction
Introduction
7 New technologies are powering the IoT, such as the LPWAN protocols, the low consumption microprocessors, the small size batteries, etc. However, there are some basic requirements to achieve good IoT deployments like interoperability, scalability, security and the cost, that are not so easy to implement, in fact, there is no standard solution yet.
The present project tries to solve part of these issues to ease the IoT deployments. Specially this project is focused on the scalability and the interoperability, also taking into account the cost.
The aim of this project is to develop a solution for facing the scalability issue in the gateway-based deployments, focusing on the management of a big number of gateways in a remote way at the same time. In addition, another aim of this project is to face the interoperability issue allowing the gateway to manage several LPWAN protocols. Besides, the previous objectives will be supported by the capability to deploy processes in an easy and remote way and without restarting the gateway. Since an approach with these features could be so expensive, a mandatory condition for this proposal is to give a solution using open source software and open hardware.
The solution must not be linked to a specific IoT vertical, then, it must be flexible and transversal to cover as many use cases as possible. In order to achieve such a dynamic solution, the designed architecture must be modular, allowing the deployment of different software components with different applications, giving the gateway the possibility to change its behaviour, allowing the gateway to implement functionalities related with edge computing like aggregation of measurements values, apply a machine learning algorithm inside the gateway for taking decisions locally, detecting alert situations or accept new IoT devices with proprietary protocols.
The management of the massive configuration will follow an event driven protocol. This kind of protocols allows the gateways to join to the system only knowing the address of the broker, no further information is needed. This provides the flexibility needed for the system in terms of communication.
The objectives of this project can be summarized in:
1. To face the problem of the scalability: to manage a big number of gateways in a remote way at the same time.
Introduction
8
2. To face the problem of the interoperability: to allow the gateway to manage several LPWAN protocols.
3. To face the reusability and flexibility: to be able to deploy components to perform specific processes in an easy and remote way and without restart the gateway.
1.1. Thesis Outline
This remainder of this document is composed of four sections:
• Section 2, State of the art: A brief view of the current status of the IoT and the gateway based architectures.
• Section 3, Description: The design and implementation of a solution that achieves the main objectives and the most part of the requirements.
• Section 4, Experiments and Use Cases: A brief view of the implemented system and the test done over the system.
• Section 5, Budget: A detailed cost of the proof-of-concept.
• Section 6, Conclusions and Future Work: conclusions about the usefulness and a critical view of the system resultant and future enhancements of the system.
2 State of the art
State of the art
11 Nowadays, the idea of IoT can be applied to almost any context of our daily life, every big company has at least a department in which a group of engineers tries to solve problems using the potential of the IoT technology. However, there is not only one kind of IoT and the technology implemented depends on the specific use case and the environment conditions (indoor, outdoor, coverage, etc.).
The IoT field covers every situation that the human could imagine; it is possible to find an IoT technology or a combination that fits perfectly for the solution needed. It does not matter if it is a nuclear power station or a park in a city, IoT has a solution or at least the tools to solve the problem or fit the improvement project.
With this idea on mind it should be possible to address any project without too much effort, but the reality is not exactly this. It is true that the current state of the technology and its cost have permitted to start R&D IoT projects and real-world deployments during the last years but mostly as a simple prototype or a proof of concept [8] [9] [10] [11] [12] [13]. This situation responds to the difficulty to manage the security, interoperability, scalability, maintainability, etc.
Some of these issues have its solution on the hardware part [14], others on the software, and others on both. Security is the most important issue of IoT [15]. The idea of having amounts of devices without protection will provoke nightmares to more than one engineer and also to the investor, the CEO of the company, and everyone involved in the project. But also to the rest of the world, because the situation could be used by hackers to attack devices and provoke, for example, a DDoS [16] [17].
Interoperability is other of the main issues of the IoT deployments. As it has been said before, sometimes the solution is composed by several technologies that need to be integrated each other. Sometimes, several companies are collaborating to develop an end-to-end IoT project that involves several parts of the IoT value chain, a first solution was given years ago and now exists the need to extend the scope of the solution having to guarantee backward compatibility with parts of the previous system in order to assure robust working of the new system.
In this situation, the interoperability is needed to be able to coordinate different kinds of devices and services characterized by their communication protocols and programming paradigms. In the IoT field the gateway has an important role for the interoperability since it works as a translator between two languages but translating between two physical protocols as it can be seen in Figure 1.
State of the art
12
Figure 1. Generic Gateway Architecture
Scalability is the most important issue from the economics point of view.
Scalability means replication, multiuser systems, the capability to manage big quantities of users and devices at the same time, etc., but, in the end, it means to develop once and use it nearly infinite times. This idea is similar to the microprocessor programming or the devices manufacturing, but in this case the scalability is applied to the Wireless Sensor Network. This means that it is possible to add new devices without changing the network configuration and with a limited effort to integrate it.
The auto-registration is one of the main features for easing the scalability, this is very related with the security because of the authentication of the devices. Another enabler for the scalability are the event-driven protocols, which allows to share messages between several devices only knowing the address of the broker, so it is possible to add a new device to the message exchanging without modifying the system. The devices are subscribed to topics and they receive the messages published on this topic by other devices. It is a very flexible communication method a largely used in IoT. An example of this technology is MQTT protocol.
Another technology, that is independent from the IoT, is growing and becoming more and more important for applications and automated systems. This technology is the Artificial Intelligence (AI). Usually it is applied on the servers due to the power consumption and the computation resources needed, but during the last five years, the idea of deploying AI on devices is taking strength and there are many companies like Microsoft or Amazon, and researchers groups working on it [18].
The main devices in charge of this task are the gateways. A gateway usually is connected to the electricity network, so its consumption is not so important and the
State of the art
13 capabilities are not so constrained like in an end IoT device. In this situation, the management of the gateways turns mandatory for maintaining or modifying the AI. The flexibility and the scalability are very important points for the deployment of these systems in a proper way. The main companies of the world are working on its own solutions for this paradigm [19]. However, one of the most important characteristics of the IoT society is the usage of open-source solutions. Hardware platforms like the Raspberry Pi, Arduino or ESP32 and software languages like Python, Java, Javascript, Node-JS are the principal enablers of the IoT.
There are implementations to transform the Raspberry Pi into almost everything, media centers, smart TV, server, video-gaming station, etc. And also implementations for the IoT environment like LoRa gateway, MQTT broker, IoT device, local server to manage the information received from IoT devices, etc. These implementations are not only from particulars, the companies are also working on its solutions for edge computing like Microsoft or Amazon, or giving the Raspberry Pi the capabilities of an industrial gateway like Eurotech.
Eurotech bases its framework on Eclipse Kura that is an extensible open source IoT Edge Framework based on Java/OSGi included on the IoT Eclipse Project [20]. Kura supports several platforms like Raspberry Pi 2-3, Intel UP2, and Rock960.
Kura Eclipse, as it is based on Java/OSGi, is programmed in modular software, this permits to deploy parts in an independent way and without restart the whole system.
Kura manages the firewall, the connectivity of the device (WiFi, Access Point, Ethernet), and provides services to control peripherals like the Raspberry Pi GPIO. Kura framework is composed by several bundles, more than 150, and also provides a H2 database engine [21] and an MQTT broker that is used as the main way of communication among bundles.
The open programming languages are easing the task to develop a simple dashboard or even a complete application. This is a true advantage for demos and prototypes, because it is not needed to invest lots of time in the development of the application, user interface or dashboard. This kind of languages, like Node.js, JavaScript, python, etc., have implemented most of the libraries that could be needed for the development of an application, so the programmer does not have to program them, this saves lots of time again. From this environment of programming languages not so strict like Java or C, appears Node-RED [22].
State of the art
14
The IoT environment does not stop growing, everybody is working on it and what today can look like a dream, tomorrow it could be real.
3 Description
Description
17 This section contains the explanation of the design and implementation.
3.1. System Requirements Specification
The requirements of the system are the following:
1. IoT Gateway.
a. Modular software. The gateway must change the configuration, processes, drivers, etc. So, it is needed a modular software to be capable to change modules without restarting the gateway.
b. Flexible communication protocol to interact with the cloud. The communication protocol must be able to add new devices and transport a message in broadcast.
c. Low cost open hardware device. The cost of a WSN is the main impediment to implement this kind of nets. Open hardware device avoids the dependency with third parties, being a fully replicable device.
d. Connectivity and configuration for the LPWAN interfaces. The gateway must have the possibility to use its embedded interfaces or the external ones. These interfaces must be plug and play and, when possible, configurable.
2. Management tool for managing the gateways.
a. Flexible database. The data sent by the gateways must be stored for a posterior processing and to support the graphical representation. The database must be flexible to store the different messages that will come from the different sensors.
b. User interface. The data stored, the incoming and the configurations must be represented in a graphical way to ease the task to maintain the gateways.
Description
18
3.2. System Design
3.2.1. Architecture Overview
Figure 2. Architecture Overview
The architecture represented in Figure 2 defines the whole system architecture from a very high level point of view. The design is based on the typical gateway-based systems, see Figure 1. It proposes some enhancement from the existing systems.
Current gateways translate only one protocol into another, also it is possible to interchange the interfaces to change one of the protocols but it is always a one-to-one relationship.
This gateway, named Rosetta because of the name given to the first bilingual document discovered [23], has the capability to transform several protocols at the same time into others. Additionally, provides logical layers to be able to manage important issues, for example, updating drivers for new devices connected to the gateway, deploying modules for data analytics, or scheduling radio links to update firmware of sensor devices via OTA techniques.
The purpose of this design is to provide a flexible platform that works in almost any situation with the possibility to manage many Rosetta devices at the same time in an easy way. Besides, the Management Tool can deploy components to perform processes to give Rosetta capabilities of an edge computing device.
The architecture is divided into two parts: Rosetta and Management Tool that are described in the next sections.
Description
19
3.2.2. Rosetta Architecture
Figure 3. Rosetta Architecture
As it can be seen in Figure 3, Rosetta is divided into two different parts: hardware and software, and it proposes enhancements in both fields.
Hardware
Rosetta can fit to the typical gateway architecture and provide one interface for communicating to the sensors, and other interface for communicating to the Internet.
Additionally, and this is the enhancement, Rosetta can have several interfaces for both links. This means, it can receive information from 802.15.4 devices and, at the same time, from LoRa, Bluetooth Low Energy, etc. Furthermore, Rosetta can send this information to the internet using interfaces for Ethernet, NB-IoT, 3G, etc. or it could be a part of a bigger network and to use a LPWAN protocol to reach the next Rosetta Gateway in case that the first Rosetta does not have connectivity.
Software
Rosetta is based on modular software, so it is possible to deploy programs on it without needing a restarting of the system. This allows to deploy drivers for the interfaces only when needed, leaving the memory as free as possible.
Description
20
The modular software provides flexibility during the development and also during all the stages of the project, being possible to modify the system configuration or the system performance when it is installed.
The configuration of the system and all the data needed is stored using the persistence module.
3.2.3. Management Tool Architecture
The management tool consists of two parts, as can be seen in Figure 4: user interface and backend. Both parts work together to provide the system control, the downlink traffic: system commands, and the data recovery, uplink traffic: data from the gateways.
Figure 4. Management Tool Architecture
User interface shows the data received from the gateways, the commands to be send and the current status of the system.
The backend manages the information received from the communications module and stores the data into a database using the persistence module. It also manages the information to be shown on the user interface.
Description
21
3.2.4. Functionalities Sequence Diagrams
3.2.4.1. Data Upload
The sequence diagram represented in Figure 5 describes the process to upload data from the sensors from the Rosetta point of view. The data from a Wireless Sensor Network (WSN) arrives to the gateway by interface 1, the message is sent to the interface 1 module where it is decoded and sent to the Gateway Core that decides which is the message destination. The message is sent to interface 2 module and is sent to the Management Tool.
Figure 5. Data Upload Rosetta Gateway Point of View
Figure 6 describes the process on the Management Tool, decoding the payload and storing it at the same time that it could be visualized.
Figure 6. Data Upload Management Tool Point of View
Description
22
3.2.4.2. Component Management
In Figure 7 it is represented the procedure to update/install a component from the point of view of the Management Tool. Ignoring the component repository section, it is also valid for start, stop and uninstall.
The System Manager selects the destination, it could be several Rosetta gateways, and the message is built to be sent by the communications module. This message arrives to Rosetta, where the component to download will be requested to the component repository and installed when downloaded. Rosetta sends notifications during the installation process.
Figure 7. Component Installation/Update Management Tool Part
Figure 8 describes the installation/update process from the point of view of the Rosetta gateway. A message comes from the Management Tool, this message is treated by the interface 2 and its module. The gateway core gets the message information and requests the file to install. The gateway core notifies the status of the process.
Description
23 Figure 8. Component Installation/Update Rosetta Gateway Part
3.2.4.3. Rosetta Configuration
The Figure 9 describes how Rosetta get the configuration from the persistence module and applies it.
Figure 9. System Start Configuration.
3.2.4.4. Commands sending
In Figure 10, the procedure to change configurations of the Rosetta Gateway is described. The procedure is the same than the procedure of Figure 7, but in the Gateway
Description
24
Core the changes are saved and the commands applied instead of initialize an installation.
Figure 10. Command sending
3.2.5. Databases Definition
3.2.5.1. Rosetta Gateway Database
A simple database with the information needed to configure the gateway. The relational diagram is shown in Figure 11.
• ROUTER_TABLE: It contains information to route the messages that arrives to the Message Router.
o ID: Route identification.
o INPUT_DRIVER: The driver that has sent the message.
o OUPUT_DRIVER: The driver that must receive the message.
• SERIAL_COMM_CONFIG
o ID_DEVICE: Interface identification.
o ID_DRIVER: Identifies the driver that must manage the messages coming from this interface.
o BAUDRATE: Baudrate for the serial communication configuration.
o DATA_BITS: Data size in bits.
Description
25 o STOP_BITS: Number of bits to determine the end of the
communication.
o PARITY: An error control mechanism based on the parity of the bits sent.
o FLOW_CONTROL: Kind of flow control.
o PUB_OPTION: It specifies if the data must be sent to the cloud, stored in data based, both at the same time or none.
• DRIVERS_CONFIG: Relates the driver data with a transformer.
o ID_DRIVER: Driver that sends the message.
o ID_TRANS: Identification of the transformer that must be applied to the data.
• MQTT_CONFIG: The configuration of the MQTT clients.
o ID_CONNECTION: Identification of the connection.
o USER: User for the client.
o PASSWORD: Password for the client.
o URL: The URL of the MQTT Broker.
o PORT: The port of the MQTT Broker.
Description
26
Figure 11. Gateway Database
3.2.5.2. Management Tool Database
This database is installed together with the Management Tool and it is accessible from the backend Management Tool. It contains two collections:
• Gateway_data: Saves the messages with data sent by the gateways.
• Gateway_id: Saves a list of the gateways that have sent data and its current status.
3.2.6. Information Model
The system needs a standardized message to be understood by every part of the system. The format of the message is based on JSON, as can be seen in Figure 12, and its named Rosetta Mesage:
• timestamp: Timestamp in milliseconds.
• Metrics: An object that contains information.
• Message: A string that represents a message. This data has different format when is a command than when is a data message. In a data message it is the hexadecimal representation of the data field. In the case of a command message, this field contains a JSON codified in Base64.
Description
27 This JSON could be for managing bundles or change the gateway configuration depending on the action field.
• idGateway: Identifies the gateway, even if it is the origin or the destination.
• idDevice: Identifies the interface, even if it is the origin or the destination.
• Origin: Identifies the bundle that sent this message.
• Status: Identifies if the message has been routed or not. A message is routed when it passes by Message Router bundle.
• pubOption: specifies if the message is for being sent, stored, both o none.
• Action: specifies the task to be done with this message. It is only used for commands.
• Data: contains the information retrieved from the sensors.
• Temperature: temperature value in Celsius Degrees.
• Humidity: humidity percentage measured.
Figure 12. JSON Rosetta Message
Figure 13 represents the message used by Rosetta to manage the components.
This message is encapsulated in the message field of the Rosetta Message.
{
“timeStamp”:number,
“metrics”:{
“message”:"string", “idGateway”:"string",
“idDevice”:"string",
“origin”:"string", “status”:”string”,
“pubOption”:number,
“action”:”string”,
“data”:{
“id”:number,
“Temperature”:number,
“Humidity:number, }
} }
Description
28
• request.id: id for the request.
• requester.client.id: client Id that makes the request.
• job.id: an id for the specific job.
• dp.uri: uri for download the deployment package (bundle compressed).
• dp.name: The name specified on the deployment package project.
• dp.version: The version specified on the deployment package project.
• dp.download.protocol: The protocol to download the deployment package.
• dp.install: Specifies if the deployment package must be installed after the download.
• dp.install.system.update: Specifies if it is a system update.
Figure 13. Command Message Format for Components Management
Figure 14 describes the message that modifies the serial communication configuration of a specific device. The message must be only the information desired to change and the idDevice. This message is encapsulated in the message field of the Rosetta Message.
{
"metrics":
{
"request.id":"string",
"requester.client.id":" string",
"job.id":number,
"dp.uri":" string",
"dp.name":" string",
"dp.version":"string”,
"dp.download.protocol":" string",
"dp.install":boolean,
"dp.install.system.update":boolean }
}
Description
29 Figure 14. Command Message Format for Serial Configuration Update
3.3. Implementation Design
In order to test the proposed design, a proof-of-concept has been developed.
Referring to the section 3.1, several implementation decisions have been made and are shown in Table. I:
Environment Design Requirement Implementation Resource
IoT Gateway
Modular Software Java OSGi - Kura
Flexible communication protocol MQTT Low cost open hardware device Raspberry Pi Connectivity and configuration for
LPWAN interfaces
FiPy and Waspmote connected via USB
Management Tool Flexible Database MongoDB
User Interface Node-RED Dashboard
Table. I. Requirements Compliance
The central part of this proof-of-concept is Kura that is implemented in Java OSGi, so it is important to know how is the lifecycle of an OSGi bundle, represented in Figure 15, to understand the behaviour of the implemented ones.
{
"idDevice":"string ",
"baudrate":"string",
"dataBits":number,
"stopBits":number,
"parity":"string",
"flowControl":"string",
“driver”:”string”,
“pubOption”:number }
Description
30
Figure 15. OSGi Bundle Lifecycle
Figure 16 describes the implemented system architecture, specifying the Rosetta Gateway architecture. Every communication between the Kura bundles uses the internal MQTT broker. This provides flexibility on the communication and permits to substitute one bundle for other without having coupling issues.
The Waspmote is simulating measurements and sending them by the USB connection. The data is sent in bytes and contains an Id, temperature in Celsius degrees, and humidity, ten bytes total.
The CommunicationManager Bundle, manages the USB drivers and opens the serial port connection, one for the Waspmote, another for the FiPy. The aim of this bundle is to provide communication between Kura and the interfaces.
The message received by the serial port, in bytes, is sent by MQTT to the Waspmote Driver, that has the information needed to understand the message.
The Waspmote Driver builds the RosettaMessage, that is based on KuraMessage, and sends it to the MessageRouter bundle.
The MessageRouter checks the router table using the origin of the message and get the destination of it. From this step, there are two possible situations, both have been implemented:
• Ethernet interface: Using the operating system drivers, the Inet RPI Driver connects to the external MQTT broker to publish the message.
Description
31
• NB-IoT: The MessageRouter sends the message to the FiPy Driver, where the specific command is built in the RosettaMessage. This message is sent to the CommManager. Then, it sends the message to the FiPy that is connected to the external MQTT Broker using NB_IoT and publish the message.
The bundle installations its only implemented on the Ethernet option.
Figure 16. Rosetta Gateway Implementation
Figure 17 describes the Management Tool implementation architecture. The messages are received in the MQTTClient. The message is treated by the backend core and stored on the database at the same time that it is drawn on a chart on the user interface.
From the user interface is possible to send commands to modify the existing bundles. Also is possible to see historical data of a specific gateway.
The component repository has the deployment packages that contain the bundles that can be installed on Kura.
Description
32
Figure 17. Management Tool Implementation
3.3.1. Functional Diagrams
3.3.1.1. MessageRouter
MessageRouter is a bundle in charge of forward the message to the proper destination. The destination is obtained from the database. Also, the MessageRouter saves the messages in the database in the table named Messages represented in Figure 11. The functional diagram is represented in Figure 18.
Figure 18. Functional Diagram MessageRouter Bundle
Description
33 3.3.1.2. Serial Communications Manager
The main functionality of this component is to establish and maintain the serial communication. The data received from this manager is sent to the proper driver to be understood. Also, this manager receives data from the drivers to be sent over the serial port. The functional diagram is represented in Figure 19.
Figure 19. Functional Diagram CommManager Bundle
3.3.1.3. Drivers
The drivers are bundles that implement the programming needed to understand the message coming from the interface and to encode the message in a way that the interface can understand. As the interfaces are not static, the driver can be updated, installed or deleted in remote. The generic driver functional diagram is represented in Figure 20. Figure 21 represents the particular functional diagram for the managing of the internet interfaces of the Raspberry Pi.
Figure 20. Generic Driver Bundle
Description
34
Figure 21. Internet Raspberry Pi Driver
3.3.2. Class Diagrams
DatabaseManager is in charge of accessing the gateway database to retrieve the gateway configuration.
MqttRosettaClient is a customization of the MqttClient adapted to the needs of the system.
Figure 22. DatabaseManager and MqttRosettaClient
Description
35 RosettaMessage class is the representation of the information model. It is a structure that is capable to manage the information of every component of the system.
Metrics defines the fields that the RosettaMessage can contain, including the Data class that will have the data from the sensors.
Figure 23. MessageRouter
Description
36
SerialConfig represents the configuration of the serial ports that will be connected to the main device. This information is stored in database to be recovered when the system restarts.
Figure 24. SerialConfig
Description
37 SerialComm puts together the configuration of the serial port and the serial port itself.
Also includes two classes that implement the read/write operations. These two classes are based on the class SerialPort, that is an existent class that implements the serial port operations like open, close, read, etc.
This serial port is opened using the information of the UsbService, that detects the devices connected and gets information about the devices that is checked with the configuration to open the serial port or not.
Figure 25. SerialComm
Description
38
CommManager uses the previous classes represented in Figure 22, Figure 23, Figure 24, and Figure 25. Also includes the MqttClient interfaces for receiving notifications of MQTT. The class ConfigurationUpdater allows to update a serial port configuration totally or partially. UpdateMessage interprets the message coming from the upper classes.
Figure 26. CommManager Bundle
Description
39 Driver uses the previous classes represented in Figure 22, and Figure 23. It implements the interfaces for receive notifications from MQTT. The drivers must access to the database to load their configurations. Drivers use the RosettaMessage format for the interaction among components. A special implementation
Figure 27. Driver
Description
40
MessageRouter class, represented in Figure 28, uses the previous classes represented in Figure 22, and Figure 23. It implements the interfaces for receive notifications from MQTT. The MessageRouter must access to the database to load its configuration and to load the route table. MessageRouter use the RosettaMessage format for the interaction among components.
Figure 28. MessageRouter
3.3.3. Development Resources
The implementation of this project involves several technologies, frameworks and devices, the most of them are open.
3.3.3.1. Interface Development Environments
Several IDE’s have been used for developing the different parts of this projects:
• Eclipse IDE: This framework allows to program for almost every programming language, but in this case the programming language
Description
41 chosen was Java SE 8 and OSGi. This framework is used for developing the OSGi bundles. Also, mToolkit plugin has been used for managing the bundles during the development.
• Microsoft Visual Studio Code: For programming the FiPy using the plugin Pymakr.
• Waspmote PRO IDE: For programming the Waspmote to act as an interface.
• Node-red: Used to develop the application to receive the data and send the commands.
3.3.3.2. Programming Languages
The programming languages used to code the implementation of the proof-of- concept are the following:
• Java SE 8 for the bundles programming.
• Node Js for the Management Tool programming.
• MicroPython for the FiPy programming.
• C for the Waspmote programming.
3.3.3.3. Hardware Devices
The hardware devices that have been used for the implementation are the following:
• Raspberry Pi 3B as the Gateway core.
• Libelium Waspmote as the WSN interface.
• Pycom FiPy as the Internet interface.
3.3.3.4. LPWAN Protocols
Although there are several LPWAN protocols mentioned and taken into account for the design, there is only one LPWAN protocol that has been used for the development of this proof-of-concept that is NB-IoT.
Description
42
3.3.3.5. Databases
Two kind of databases have been used for this development:
• MongoDB for the Management Tool database.
• H2 database engine for the Rosetta Gateway database.
3.3.4. Implementation Details
3.3.4.1. Publish-Subscribe-based message protocol
The protocol used is MQTT. MQTT is a very flexible protocol that permits to change the bundles without stopping the system, because there is not a dependency in the connection between the sender and the receiver. Also is possible to maintain the messages until a client subscribes to it or wait for a confirmation of reception.
This protocol has been used on the gateway for the communications among bundles, Kura does it by default among their bundles, and also for the communication with the Management Tool. Using MQTT by this way, allows to keep the gateways secure, with no open ports or connections unnecessarily, only needs to subscribe to the external MQTT broker and to download the bundle update files. With this configuration, the gateway is not exposed to the public net. The topics subscriptions and publications are specified in Figure 29. It is not defined yet because the system has been tested only with one Rosetta Gateway but, all the Gateways will be subscribed to a broadcast topic in which the commands will be sent to everyone.
Description
43 Figure 29. System Topics
3.3.4.2. Databases
Two kind of databases are needed in this system. There is one database on the cloud, this is a MongoDB database, this allows to save different data from different devices without change the database. This database is used for saving the data received on the Management Tool, the devices and their status. The implementation of this databased follows the design of the section 3.2.5.2.
Kura installs a H2 engine database that is a SQL database. It is accessible using JDBC (Java Database Connectivity). This database stores the basic configuration of the gateway, like the serial port configuration, the MQTT configuration, the messages sent, etc. The implementation of this databased follows the design of the section 3.2.5.1.
3.3.4.3. Interfaces
Hardware devices that implement communication protocols. The data is sent to the interface by the IoT devices to reach the gateway. Moreover, interfaces could be connected to the internet, being the way out to the cloud. The interface must have only one of these roles.
Description
44
3.3.4.4. Semi Plug and Play
It is necessary to update the Rosetta Gateway database before connecting a new USB interface, once this is done, the interface can be connected and disconnected without extra configurations. As part of the CommManager bundle, an USB event listener has been applied to detect new connections and disconnections, and a code has been implemented to manage the serial port of the device.
3.3.4.5. Management Tool
The application is very limited because it is only for testing the implementation of the design.
The application saves the data received from the gateway into the database, shows the data filtered by gateway identification and allows to send commands for check, start/stop or update the bundles of the gateway.
Figure 30 shows the Commands Screen. From this point of the Management Tool the System Manager is able to manage the bundles of the system. It is possible to ask for the installed bundles and their status, and install, uninstall, updated, start or stop a bundle.
Figure 30. Commands Screen
Description
45 The Figure 31 permits to see the data uploaded on live or the data stored in database.
Figure 31. Data Display Screen
The screen of Figure 32 shows the devices list and the status of each one.
Figure 32. Gateway Status List
4 Experiments
and Use Cases
Experiments and Use Cases
49
4.1. Experiments
The experiments have been done using the system of the Figure 33 which is describe in Table. II.
Figure 33. Proof-of-Concept
Identifier Description
1 Raspberry Pi 3B – Rosetta Gateway Core
2 Libelium Waspmote – Rosetta Gateway WSN Interface 3 Pycom FiPy – Rosetta Gateway Internet Interface 4 System Power Source
5 Ethernet Connector – Internet Interface
6 FiPy Power Source and Serial Port Connection 7 Waspmote Power Source and Serial Port Connection
8 NB-IoT Antenna
Table. II. Proof-of-Concept Components
Experiments and Use Cases
50
4.1.1. Semi Plug and Play Test
The interface can be connected and disconnected without restarting the system.
The effect of this feature can be observed on the charts of the Figure 34. The device has been disconnected for around thirty seconds and reconnected. The gap between the fifth measurement and the sixth is 37 seconds, this is the period of disconnection.
Figure 34. Plug and Play Test
4.1.2. Driver Installation
Installation of a bundle from zero. In Figure 35, the installed bundle drivers are shown.
Figure 35. Default Bundles
Experiments and Use Cases
51 Figure 36 the command download bundle is sent. The Waspmote driver is installed on the Rosetta Gateway.
Figure 36. Waspmote Driver Installation
Figure 37 shows the Waspmote driver installed.
Figure 37. Current Installed Bundles
Figure 38 shows an additional step to stop the bundle is added to show the capabilities of the system.
Experiments and Use Cases
52
Figure 38. Waspmote Driver Stopped
4.1.3. Data Upload Using Ethernet
With the system working, the data uploaded, using the embedded Ethernet interface, have been observed in Management Tool to test the normal behaviour. As the data is arriving to the Management Tool, it is represented on the screen as can be seen in Figure 39.
Figure 39. Dynamic Table and Charts
4.1.4. Data Stored in Database Representation
Figure 40 shows the data stored in database for the specific device selected.
Experiments and Use Cases
53 Figure 40. Static Table and Charts
4.1.5. Data Uploading Using NB-IoT
The system has been tested using the NB-IoT interface as the way out of the data from the gateway. This configuration only allows to upload data, because the commands sending is not implemented yet over NB-IoT.
4.1.6. Stress Test
The simulator of the Waspmote have reduced the delay time between messages.
The system works well with less than 9 messages per second, see Figure 41.
The implementation can be improved and reach higher throughputs.
Experiments and Use Cases
54
Figure 41. Stress Test
4.1.7. Continuous data upload test
The system has been tested during more than four hours, 13:28:29 to 17:52:22, to test the stability as can be seen in Figure 42 and Figure 43.
Figure 42. Continuous test beginning
Experiments and Use Cases
55 Figure 43. Continuous Test ending
Experiments and Use Cases
56
4.2. Use Cases
The following are not specific use cases but situations where the Rosetta Gateway implemented in the previous section could help to give a solution.
The first use case represents a situation with Ethernet connectivity and the possibility to need protocols as LoRa, if the application needs to cover long distances, ZigBee, if the application needs short range and does not need very fast communications, or BLE, if the application needs short range but high speed communications, for example. The graphical representation can be seen in Figure 44.
Figure 44. Use Case 1
The second use case, Figure 45, represents a situation where there is not Ethernet or WiFi connectivity and the 3G connectivity does not reach the place of the Rosetta Gateway. In this case, the NB-IoT connectivity, although is not so fast as 3G, could fit an application with low data rate. Also, in future works, the Rosetta Gateway could implement 3G and NB-IoT and select when to use one or another depending on the coverage or even depending on the functionality.
Figure 45. Use Case 2
5 Budget
Budget
59 Budget for the proof-of-concept developed is shown in Table. III.
Resource Units €/Unit Cost
Raspberry Pi 3B 1 32.67 32.67€
Libelium Waspmote 1 240.40 240.40€
Pycom FiPy 1 61.19 61.19€
NB-IoT card 1 - -
Data - - -
Laptop 1 1,000 1,000€
Development 480h 6 2880€
Technical University of Madrid MQTT Broker 1 0 0€
Total 4213.86€
Table. III. Proof-of-Concept Budget
6 Conclusions
and Future Work
Conclusions and Future Work
63
6.1. Conclusions
The design of Rosetta provides functionalities that are a possible path for solving the issues of scalability and interoperability. The proof-of-concept developed does not implement all the parts of the architecture, but allows testing the viability of the idea, that is valid in a first stage and needs to implement the rest of the features to test the whole system.
The current system allows to change the interface and, therefore, the communication protocol to interact with the WSN and also to interact with the Internet.
Rosetta Gateway is sending simulated data from a Waspmote and this device is able to implement LoRa, ZigBee, Bluetooth, etc. Only adding the implementation of the protocols on the Waspmote, the Rosetta Gateway will change its interface.
The upload of the data has been done by two different ways, but, using the FiPy capabilities, it implements on the same board SigFox, LoRa, Nb-IoT, LTE-M, WiFi and Bluetooth, the interface can change with almost no effort.
The commanding of the Rosetta Gateway, from the Management Tool, over Ethernet interface has shown that is possible to deploy bundles with the system running without having to restart the Rosetta Gateway. For this project, the updateable bundles are the drivers, also it is possible to update the management bundles. By using the same procedure, it is possible to deploy bundles with specific functionalities. Besides, the use of the MQTT protocol contributes to improve the scalability of the system, the Rosetta Gateways receive the command sending it only once from the Management Tool due to the Gateways subscription to a broadcast topic.
The project has shown that is possible to design and implement a system that helps the developer and the engineers to implement the application they want to, but it needs to be enhanced to become a viable solution for implement real systems.
6.2. Future Work
This proof-of-concept has tested several features and has indicated the path to follow, but this is not a final solution and it needs to be enhanced to test all the features proposed on the architecture. These are the next steps for the project.
Conclusions and Future Work
64
• To provide security. This proof-of-concept does not provide security to any parts of the system since it is trying to test the functionality. Provide security is needed and is addressed for the future work.
• To implement the remote configuration for interfaces, sensors, and bundle parameters. Sometimes, the sensor will be configurable, this feature will be integrated on the system to manage it in remote. Also, the interfaces and the bundle will have configurable parameters and these must be managed in remote.
• To implement the commands sending through NB-IoT (downlink). The connectivity using NB-IoT is only used for upload data (uplink), this does not allow to manage the system.
• To implement the transformer bundle. The Rosetta Gateway is part of a bigger project that involves deep learning and machine learning techniques. A bundle will be developed implementing a model to process the data, that will be obtained from the H2 database, and train with this data on the Rosetta Gateway.
• To implement multi-LPWAN-interface to provide the connectivity with several WSN at the same time.
• To test the multi-management feature. Although the system is designed for support the management of several devices at the same time, this is a feature that have not been tested because of the available devices.
• To provide a complete application for the management of the gateways and the data visualization taking into account the sensor identification, the interface identification and the gateway identification.
• To provide the capability to create groups of Rosetta Gateways to manage them at the same time and with the same configuration.
7 References
References
67 [1] D. Hujo, D. Schütz and B. Brunner, “Dynamic Generation of Shop Floor Gateway
Configurations from Standard System Engineering Models during Runtime,” in IEEE 14th International Conference on Automation Science and Engineering (CASE) pp.630-633, 2018.
[2] K. Dolui and C. Kiraly, “Towards Multi-Container Deployment on IoT Gateways,” in IEEE Global Communications Conference (GLOBECOM) pp. 1-7, 2018.
[3] V. K. a. R. Kirichek, “The Heterogeneous Gateways in the Industrial Internet of Things,” in 10th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT) pp. 1-5, 2018.
[4] I. Eclipse Foundation, “Kura,” [Online]. Available: https://www.eclipse.org/kura/.
[Accessed 2019].
[5] R. P. Foundation, “RaspberryPi,” [Online]. Available:
https://www.raspberrypi.org/documentation/hardware/raspberrypi/. [Accessed 2019].
[6] Libelium, “Libelium Waspmote,” [Online]. Available:
http://www.libelium.com/products/waspmote/. [Accessed 2019].
[7] Pycom, «Pycom FiPy,» [En línea]. Available: https://pycom.io/product/fipy/. [Último acceso: 2019].
[8] A. Hamied, A. Mellit, M. Zoulid and R. Birouk, “IoT-based experimental prototype for monitoring of photovoltaic arrays,” in International Conference on Applied Smart Systems (ICASS), 2018.
[9] M. Fraifer and M. Fernström, “Smart car parking system prototype utilizing CCTV nodes: A proof of concept prototype of a novel approach towards IoT-concept based smart parking,” in IEEE 3rd World Forum on Internet of Things (WF-IoT), 2016.
[10] S. Nnedimpka Nnadi and F. E Idachaba, “Design and Implementation of a Sustainable IOT Enabled Greenhouse Prototype,” in IEEE 5G World Forum (5GWF), 2018.
References
68
[11] D. Lestari, I. D. Wahyono and I. Fadlika, “IoT based Electrical Energy Consumption Monitoring System Prototype: Case study in G4 Building Universitas Negeri Malang,”
in International Conference on Sustainable Information Engineering and Technology (SIET), 2017.
[12] G. D. Salazar Ch, C. Venegas and L. Marrone, “MQTT-Based Prototype Rover with Vision-As-A-Service (VAAS)in an IoT Dual-Stack Scenario,” in Sixth International Conference on eDemocracy & eGovernment (ICEDEG), 2019.
[13] C. Calderon-Cordova, M. Gonzaga, J. Morales, M. Morocho, B. Torres and C.
Ramirez, “Prototype industrial IoT applied to temperature monitoring in storage silos of dairy products,” in 13th Iberian Conference on Information Systems and Technologies (CISTI), 2018.
[14] R. Jinnai, A. Inomata, I. Arai and K. Fujikawa, “Proposal of hardware device model for IoT endpoint security and its implementation,” in IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), 2017.
[15] S. Naik and V. Maral, “Cyber security — IoT,” in 2nd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT), 2017.
[16] E. DreyFuss, “SECURITY NEWS THIS WEEK: TELEGRAM SAYS CHINA IS BEHIND DDOS,” 15 June 2019. [Online]. Available:
https://www.wired.com/story/telegram-says-china-behind-ddos/. [Accessed 20 June 2019].
[17] Z. Whittaker, “TechCrunch,” March 2019. [Online]. Available:
https://techcrunch.com/2019/05/02/ddos-attack-california-energy/. [Accessed 20 June 2019].
[18] T. Yamakami, “An Experimental Implementation of an Edge-based AI Engine with Edge-Cloud Coordination,” in 18th International Symposium on Communications and Information Technologies (ISCIT), 2018.
References
69 [19] A. W. Services, “AWS Amazon,” 2019. [Online]. Available:
https://aws.amazon.com/greengrass/features/. [Accessed June 2019].
[20] E. Foundation, “Iot Eclipse Org,” [Online]. Available: https://iot.eclipse.org/projects/.
[Accessed 2019].
[21] “H2 Database Engine,” [Online]. Available:
https://www.h2database.com/html/main.html. [Accessed 2019].
[22] O. Foundation, “node-red,” [Online]. Available: https://nodered.org/. [Accessed 2019].
[23] T. B. Museum, “The British Museum,” [Online]. Available:
https://www.britishmuseum.org/research/collection_online/collection_object_details.
aspx?objectId=117631&partId=1&searchText=YCA62958&page=1. [Accessed 2019].
