6.4.4.1 Interface testing
The major test objective of performing interface testing is:
a. - Locate errors that can prevent the system from operating at all or b. - locate errors in timing of interface responses
Interface Testing is similar to interface analysis, except test cases are built with data that tests all interfaces. The types of errors detected with this kind of testing are:
a. -Input or output description errors b. -Inconsistent interface parameters
NOTE Without the use of a CASE tool for the production of the design, or code, manual searching for interface parameters in all design or code modules can be time consuming.
Several levels of detail or completeness of testing are feasible. The most important levels are tests for:
a. all interface variables at their extreme values;
b. all interface variables individually at their extreme values with other interface variables at normal values;
c. all values of the domain of each interface variable with other interface variables at normal values;
d. all values of all variables in combination (this is only feasible for small interfaces);
These tests are particularly important if the interfaces do not contain assertions that detect incorrect parameter values.
One example of interface testing is the so-called ‘Requirements Based Hardware-Software Integration Testing’. This testing method should concentrate on error sources associated with the software operating within the target computer environment, and on the high-level functionality. The objective of requirements-based hardware-software integration testing is to ensure that the software in the target computer satisfies the high-level requirements. Typical errors revealed by this testing method include:
a. Incorrect interrupts handling.
b. Failure to satisfy execution time requirements.
c. Incorrect software response to hardware transients or hardware failures. for example un-correctable EDAC error or a “FPU exception.
d. Inability of built-in test to detect failures.
e. Errors in hardware-software interfaces.
f. Incorrect behaviour of feedback loops.
g. Incorrect control of memory management hardware or other hardware devices under software control.
h. Stack overflow.
i. Incorrect operation of mechanism(s) used to confirm the correctness and compatibility of field-loadable software.
j. Violations of software partitioning.
Another example of interface testing is the so-called ‘Requirements-Based Software Integration Testing’ that concentrates to exercise the code on the components inter-relationships. The objective of requirements-based software integration testing is to ensure that the software components interact correctly with each other and satisfy the software requirements and software architecture. This method can be performed by expanding the scope of requirements through successive integration of code components with a corresponding expansion of the scope of the test cases. Typical errors revealed by this testing method include:
a. Incorrect initialization of variables and constants.
b. Parameter passing errors.
c. Data corruption, especially global data.
d. Bad end-to-end numerical resolution.
e. Incorrect sequencing of events and operations.
6.4.4.2 Robustness Testing
Robustness testing is more focused on evaluating specific dependability aspects of the software.
Robustness testing goal is to demonstrate the ability of the software to cope with abnormal environment conditions or behaviour (abnormal inputs or catastrophic conditions). Robustness test cases include:
a. Real and integer variables should be exercised using equivalence class selection of invalid values.
b. System initialization should be exercised during abnormal conditions.
c. The possible failure modes of the incoming data should be determined, especially complex, digital data strings from an external system.
d. For loops where the loop count is a computed value, test cases should be developed to attempt to compute out-of-range loop count values, and thus demonstrate the robustness of the loop-related code.
e. A check should be made to ensure that protection mechanisms for exceeded frame times respond correctly.
f. For time-related functions, such as filters, integrators and delays, test cases should be developed for arithmetic overflow protection mechanisms.
g. For state transitions, test cases should be developed to provoke transitions that are not target computer emulator or a host allowed by the software requirements.
h. environment conditions should also be exercised to extreme or abnormal situations to ensure protection mechanisms work as expected
6.4.4.3 Performance testing
Performance testing is focused on the requirements specification that includes throughput and response requirements for specific functions, perhaps combined with constraints on the use of total system resources (e.g. CPU usage, memory usage, timing conditions). The proposed system design is compared against the stated requirements by:
a. producing a model of the system processes, and their interactions;
b. determining the use of resources by each process, for example, processor time, communications bandwidth, storage devices, etc.
c. determining the distribution of demands placed upon the system under average and worst-case conditions;
d. computing the mean and worst-case throughput and response times for the individual system functions.
For simple systems an analytic solution can be used, while for more complex systems some form of simulation should be used to obtain accurate results.
Before detailed modelling, a simpler ‘resource budget’ check can be used which sums the resources requirements of all the processes. If the requirements exceed designed system capacity, the design is infeasible. Even if the design passes this check, performance modelling can show that excessive delays and response times occur due to resource starvation. To avoid this situation, engineers often design
systems to use some fraction (for example 50%) of the total resources so that the probability of resource starvation is reduced.
Performance testing objectives are to ensure that the system meets its timing and memory requirements, and acquire measures to be taken as soon as the target operational equipment is available. Stress testing is the technique mostly used for this type of tests.