After a short motivation on the topic discussed in this thesis, the concrete problems handled here, are outlined. Then, the structure of the thesis is provided, followed by its roadmap for reading purposes.
In addition, before the main contributions are explained, a brief report on the scope is given. The major results achieved in this thesis apply to the software part of embedded systems, in general. Following the current trends of model-based development, the actual scope of this work refers to the models of software, on which, the systems are built. To avoid repeating this term, whenever system (or system model, system design, software) is referred to in the thesis, the model of a software-intensive embedded system is usually meant, unless the context is ex- plicitly indicated. This form of reasoning reflects the tendency to study the abstract level of sys- tems within the considered domain [Pre03b, Con04b, Utt05, BFM+05, CFG+05, AKR+06, BDG07]. This practice also reflects the trend that the embedded systems are often seen in a ho-
listic way, i.e., both software and its surrounding hardware trigger the expected functionality [KHJ07].
The main research problem this thesis is concerned with, relates to assuring the quality of the
embedded system by means of testing at the earliest level of its development. Based on the
analysis of the overall software and test development process, the following questions arise: 1. What is the role of a system model in relation to quality assurance? What is the role of
a test model and what elements does such a test model include? What does MBT mean in the context of embedded systems? Is it possible to use a common language for both system and test specifications?
2. How can discrete and continuous signals be handled at the same time? How should a
test framework be designed and a test system realized? What are the reasons and con-
sequences of the design decisions in terms of test generation and test evaluation? 3. How can the process of test specification and test execution be automated to the high-
est possible extent? What is / is not possible to be automated and why?
4. How can the test quality of the test method be assured itself? Which means should be used and what do they mean in practice?
The resulting contributions of this thesis can be divided into four main areas:
1. Model-based test methodology for testing the functional behavior of embedded, hybrid, real-time systems based on the current software development trends from practice; 2. In the scope of this methodology, a manner to test the behavior of hybrid systems, in-
cluding the algorithms for systematic test signal generation and signal evaluation; 3. Synthesis of a test environment so as to automate the creation of a comprehensive test
system, which is achieved by means of test patterns application that are organized into
a hierarchy on different abstraction levels;
4. Assurance of the quality of the resulting test by providing the test metrics and support- ing high coverage with respect to different test aspects.
These are denominated as challenges in the following and the discourses are tackled for each of them separately, but not in isolation.
For the first challenge, now an introductory remark should already be given. The test frame- work resulting from this thesis is called Model-in-the-Loop for Embedded System Test
(MiLEST). It is realized in the ML/SL/SF since currently about 50% [Hel+05] of functional be- havior for embedded systems, particularly in the automotive domain, is modeled using this en- vironment. Considering the fact that nowadays the integration of validation, verification, and testing techniques into common design tools is targeted [Hel+05], the argumentation for choos- ing this framework for test extensions becomes clear. This practice enables to find a common understanding of software quality problems for both system and test engineers.
In order to clarify and solve the challenges listed above, in the upcoming paragraphs the struc- ture of this work will be provided with a special emphasis on the given challenges and devel- oped contributions for each of them.
This thesis is organized as follows. This chapter gives an overview and scope of the research topics of this thesis. It introduces the problems that the work is dealing with, its objectives, con- tributions, structure, and roadmap.
Next, Chapter 2 (cf. Figure 1.1) includes the backgrounds on embedded systems, control the- ory, ML/SL/SF environment, and test engineering. Additionally, the test dimensions are ex- tracted so as to guide the general aims of this work. The emphasis is put on functional, abstract, executable, and reactive tests at the Model-in-the-Loop (MiL) level.
Chapter 3 introduces the related work on MBT with respect to embedded systems. For that pur- pose an MBT taxonomy is provided. Herewith, a link to the first challenge is done. The roles of the system model and test model are analyzed. Also, common language for both of them is ap- plied. The discussion results in a shape of the test methodology proposed in this thesis. It is called Model-in-the-Loop for Embedded System Test (abbreviated as MiLEST) and realized as an ML/SL/SF library.
All of the chapters named so far constitute the first general part of the thesis.
The second part relates to the test approach developed herewith. Chapter 4 characterizes a new means for signal description by application of signal features. By that, it relates to the second
challenge answering the question of how to handle continuous and discrete signals simultane-
ously. The algorithms for signal-feature generation and evaluation are presented. They are used along a nested architecture for the resulting test system, which is described in Chapter 5 in de- tail. Additionally, an overview of the proposed test development process and its automation is provided. A discussion on test patterns is given so as to support a fast and efficient reusability of the created test specifications. By that, the third challenge is addressed.
Chapter 5 utilizes the results of Chapter 4 and addresses the further questions of the second
challenge. Different abstraction levels of the MiLEST test system are outlined. The test harness
level including the patterns for test specification, test data generator, and test control is de- scribed. Then, the test requirements level, test case, and validation function levels follow sub- sequently. Based on that, algorithms for an automatic test stimuli generation are investigated. This relates again to the third challenge. The obtained test models can be used for both validat- ing the system models and testing its implementation.
Chapter 5 also includes the first considerations on the integration level tests. Here, benefits of applying different views on the test specification are discussed.
Finally, the last part of this thesis reveals the practical substance of the work. In Chapter 6, three case studies are discussed to validate each of the presented concepts in practice. The ex- amples are related to the functionality of an adaptive cruise control utilized in a vehicle.
Afterwards, they are evaluated in Chapter 7, which deals with the test quality metrics for the proposed test methodology, obtained test designs, and generated test cases. This piece of work relates to the fourth challenge. The concepts of test completeness, consistency, and correctness are handled herewith.
Chapter 8 completes this work with a summary and outlook. The MiLEST capabilities and limi- tations are reviewed, the general trends of the quality assurance for embedded systems are re- called and influences of the contributions of this thesis are outlined.
Fundamentals Test Approaches Test System Test Specification Test Data Generation Transformation Part II – MiLEST Part I – Introduction