Compared to the actual test, this activity is performed less frequently, but is strongly oriented to the specification of the SUT and requires much higher cognitive performance. The regeneration of test cases is usually necessary whenever the specification of the software system has changed (e.g. adding or removing requirements by the client or iterating the incremental software devel- opment) or if defects are observed in the operation of a software whose possible cause has not yet been covered by test cases.
An economic risk of manually creating test cases results from the increasing complexity of the test code with progressive development of a software product. Therefore, the generation of tests can help to control of test complexity and to reduce the effort compared to manual test case creation. To avoid economic damage from defective software, it is necessary to create high quality test cases with a risk adequate test coverage. Due to the high test complexity of software systems, this task is demanding for human actors. For example, functional areas of a software system can be inadvertently overlooked or their significance for the overall qualitative impression of a software system could be misjudged. In both cases, it is to be feared that defects in the test case specification will only be recognised late and a correction of the test case specification will require unscheduled corrective measures.
On the one hand, there is the potential to minimise human error sources by providing technical support for test case generation. Also software tools supporting a human actor in his work in such a way that frequent sources of error (e.g. overlooking requirements due to high software complexity) become less likely. On the other hand, it is also possible to speed up the revision of test cases by drawing human actors’ attention to areas of the software, where changes in the specification require adjustments to tests.
Test Coverage Criteria
Testing software systems requires the execution of the SUT under defined conditions and the interpretation of the behaviour of the SUT in order to draw conclusions about the correct im- plementation of the system specification. Software systems generally implement more than just functionality. In order to fully test a software system, all functional properties must be addressed during testing. However, this is not possible with respect to the problem addressed in this thesis, since only the RFID-specific properties of the application are considered. Nevertheless, the same metrics can be applied to the RFID-specific tests.
Coverage criteria for software tests determine which parts (functions) of a software are addressed by tests. Coverage criteria usually refer to the control flowchart of a software whose nodes represent individual statements of the source code. Coverage criteria have already been defined and examined in the scientific literature (see Patton [32], Myers et al. [10], [39]). These define four different coverage criteria:
When speaking of coverage in the context of activity diagrams, it is typically about the sys- tematic traversal and structural coverage of the directed graph that the activity diagram spans. Thus, the concrete goal is to cover individual (1) nodes (activity or actions), (2) edges (transi- tion or flow edges) or (3) paths. The latter can be restricted to certain subclasses if necessary. E.g. (1) simple paths without inner loops, (2) primary paths (corresponding to simple paths with maximum length) or (3) semantically meaningful predefined paths describing a certain user behaviour, so-called scenarios.
Testing by covering all paths is a practically or even theoretically unsolvable task, as loops can lead to an infinite number of paths. Thus, all practicable methods are based on a limited number of paths. The special constructs of activity diagrams can also prove to be problematic, which enable to model concurrency using fork and join concepts. The resulting non-determinism, in relation to the sequence of events (i. e. interleaving) in the I/O behaviour that can be observed from outside, may lead to a large number of possible constellations that need to be taken into account during testing. Sometimes, certain sequences of events can also be considered equivalent. However, these are not easy to identify without having a corresponding oracle, for example. Statement coverage testing (C0 tests) fulfils a coverage criterion that requires each statement
contained in the code of the software system to be addressed by at least one test case, i.e. executed as part of the test execution.
The criterion branch coverage, on the other hand, requires test cases for each possible alter- native sequence of the control flow at each branch. Branch coverage tests subsumes statement coverage because this criterion necessarily reaches all statements of the code. In the literature, branch coverage tests are referred to as C1 tests. Usually, this criterion is regarded as a minimum
test criterion for control flow oriented testing.
The path coverage (C2 tests) criterion requires that all possible paths of the control flow
graphs are covered by a test. However, this category is further divided into C2a, C2b and C2c
tests. C2a tests require a complete path coverage where all possible paths are tested. It is not
feasible in practice to cover C2a, because the control flow graphs may contain cycles that lead
to an infinite number of paths. To avoid this, the C2b coverage defines that cycles in the control
flow graph are executed at most twice. The C2c test coverage criterion extends the C2b in the
sense that here coverage is reached when a specified number of cycle passes are performed. In contrast to the the previous coverage tests, the condition coverage provides a different view by examining the logical structure of conditional decisions at control flow branches. These are often composed of several sub-decisions. In the terms of the C3 test, these are broken down into
atomic decisions and evaluated individually to be true or false. Similar to C2 tests, a distinction
is made between C3a (simple condition coverage), C3b (modified condition coverage) and C3c
(multiple condition coverage). Since the individual consideration of the partial conditions often leads to a combinatorial explosion, in practice a variation of C3b is applied. This variant, called
minimal conditional coverage, requires that each of the atomic predicates and the composite condition are evaluated to be true and false.