Catálogo de Especies Amenazadas de Canarias

In his thesis [31] Loft presents a solution for automated integration testing of embedded software running on remote devices. The framework described by Loft is partitioned into a host and a target module connected via an RS232 interface as shown in Figure 4.2.

The approach used by Loft employs a Remote Procedure Call (RPC) model to communicate ac- tions from the host to the target device. This is implemented by a host-side API abstraction, which maps to a device-side API. The device-side API includes several instrumentations of rel- evant methods on the target. By employing this design Loft enables the specification of tests on the host using the API abstraction, while the execution is performed remotely through RPC model and device API. Subsequent validation and reporting is performed on the host.

In order to realize RPC, Loft utilizes a lightweight protocol suitable for low-bandwidth RS232 communication. Figure 4.3 shows a packet transmitted by the host in order to invoke an action on the device. Further packages are defined which enable request and return of system flags

Chapter 4. Test Automation on Low-Resource Embedded Systems

Figure 4.2: Overview of the test framework presented by Loft.

and events relevant to assertion of the state of the system and the functionality of event-based peripherals such as the RS232 itself.

Figure 4.3: Invoke Action packet used by the solution presented by Loft.

While the framework presented by Loft is designed for remote integration testing, the principles are useful for remote test automation in general. The approach requires instrumentation to be performed on top of the AUT but it avoids Built-In-Test (BIT) code. Utilizing a standardized communication interface such as RS232 ensures that the approach can be applied to a wide range of devices while the lightweight protocol ensures minimal impact on the execution of the AUT.

4.3.3 Comparison of related work

Research focused on realizing automated testing on embedded devices is sparse and typically aimed at either traditional integration and unit testing or GUI testing of powerful embedded de- vices such as smartphones. By analyzing related work however, generally applicable approaches can be identified addressing the challenges of automated GUI testing of embedded applications described in Section 2.3.1.

The solution presented by Kervinen et al. relies on the use of a general purpose tool, m-Test, for accessing and controlling Symbian devices. The approach supports GUI testing on hetero- geneous targets sharing a common denominator, i.e., the Symbian OS. Invocation and extraction is targeted at the denominator rather than the application itself, thus abstracting away the target- specific implementation. The approach to validation by ascertaining the state of a reflection of the AUT is useful for moving the majority of processing onto the host. However, due to the reliance

Designing embedded test automation

upon bitmap transfers from the target to the host and the complex image processing employed by m-Test, the overhead introduced is infeasible for low-resource devices. Furthermore, the use of target-specific tools such as m-Test in the presented approach is practical but infeasible for non-standardized heterogeneous devices as it would require such a tool to be produced for every specific device type which presents significant development costs.

The solution presented by Loft focuses on low-overhead, packet-based communication and di- rected RPC calls to instrumented application methods. The approach benefits from the simplicity of using the RPC model but requires intrinsic knowledge of the AUT in order to introduce the needed instrumentation. By employing a common transport found on embedded devices, i.e., RS232, Lofts solution is applicable to numerous embedded devices.

Both Loft and Kervinen et al. describe a similar approach to test execution by using host-side abstractions of functionality existing on the target. Loft achieves this through RPC and Kervinen et al. through the employment of host-side keywords as well as m-Test. While they share char- acteristics in remote invocation, their individual approaches are very different. Kervinen uses a target-specific tool to control the device while Loft uses low-level communication mechanisms to send binary packets. These are in turn interpreted and matched to their corresponding target procedures.

Kervinens solution benefits from low development effort by using an existing tool. However it is limited in terms of efficiency by the computational overhead produced by continuous bitmap transfers. Lofts solution benefits from low computational overhead but is limited by development effort needed to apply necessary instrumentation to the software under test.

With regard to this thesis work, the solution by Loft is better suited for low-resource devices, while the solution by Kervinen et al. presents valuable concepts for GUI testing applications by means of a common denominator, i.e., the Symbian OS.

The communication protocol presented by Loft offers great performance efficiency by minimizing communication time between the host and target. The use of the RS232 interface also provides a high degree of transferability.

By targeting a common denominator shared by multiple devices, the solution by Kervinen et al. offers increased transferability and compatibility on related targets. However, instrumentation as described by Loft would be required. Using an RPC model may increase maintainability of the target framework but would also increase the overall code size.