El mensaje estético

The validity of the above definition of an ideal test depends on how “thor- oughly”T exercisesP. Some people equatethorough test with exhaustive or complete test, in which caseT =D.

COMPLETE(T, C): Athorough test T is defined to be one satisfying COM- PLETE(T,C), where COMPLETE is a predicate that defines how some testselection criteria C is used in selecting a particular set of test dataT

fromD. COMPLETE(T,C) will be defined in a later part of this section. Essentially,C defines the properties of a program that must be exercised to constitute a thorough test.

Reliable Criterion: A selection criterion C is reliable if and only if either every test selected by C is successful or no test selected is successful. Thus, reliability refers to consistency.

Valid Criterion: A selection criterionC isvalid if and only if whenever P

is incorrectC selects at least one test setT which is not successful forP. Thus, validity refers to the ability to produce meaningful results.

Fundamental Theorem. (∃T ⊆D)(COMPLETE(T , C)∧RELIABLE(C)∧ VALID(C)∧SUCCESSFUL(T ))⇒(∀d∈D)OK(d)

Proof. LetP be a program andD be the set of inputs forP. Letd be a member of D. We assume thatP fails on inputd. In other words, the actual outcome of executingP with inputd is not the same as the expected outcome. In the form of our notation, ¬OK(d) is true. VALID(C) implies that there exists a complete set of test data T such that ¬SUCCESSFUL(T). RELIABLE(C) implies that if one complete test fails, all tests fail. However, this leads to a contradiction that there exists a complete test that is successfully executed.

One may be tempted to find a reliable and valid criterion, if it exists, so that all faults can be detected with a small set of test cases. However, there are several difficulties in applying the above theory, as explained in the following:

• Since faults in a program are unknown, it is impossible to prove the reli- ability and validity of a criterion. A criterion is guaranteed to be both reliable and valid if it selects the entire input domainD. However, this is undesirable and impractical.

• Neither reliability nor validity is preserved during the debugging process, where faults keep disappearing.

• If the program P is correct, then any test will be successful and every selection criterion is reliable and valid.

• IfPis not correct, there is in general no way of knowing whether a criterion is ideal without knowing the errors inP.

2.2.2 Theory of Testing

LetD be the input domain of a programP. LetC denote a set of test predicates. Ifd∈D satisfies test predicatec∈C, thenc(d) is said to be true. Selecting data to satisfy a test predicate means selecting data to exercise the condition combination in the course of executingP.

With the above idea in mind, COMPLETE(T,C), whereT⊆D, is defined as follows:

COMPLETE(T , C)≡(∀c∈C)(∃t ∈T )c(t)∧(∀t ∈T )(∃c∈C)c(t) The above theory means that, for every test predicate, we select a test such that the test predicate is satisfied. Also, for every test selected, there exists a test pred- icate which is satisfied by the selected test. The definitions of an ideal test and thoroughness of a test do not reveal any relationship between them. However, we can establish a relationship between the two in the following way.

LetB be the set of faults (or bugs) in a programP revealed by an ideal test

TI. Let a test engineer identify a set of test predicatesC1 and design a set of test casesT1, such that COMPLETE(T1,C1) is satisfied. Let B1 represent the set of faults revealed byT1. There is no guarantee thatT1 reveals all the faults. Later, the test engineer identifies a larger set of test predicatesC2 such thatC2⊃C1and designs a new set of test casesT2 such thatT2⊃T1 and COMPLETE(T2,C2) is satisfied. LetB2 be the set of faults revealed byT2. Assuming that the additional test cases selected reveal more faults, we haveB2⊃B1. If the test engineer repeats this process, he may ultimately identify a set of test predicatesCI and design a set of test casesTI such that COMPLETE(TI,CI) is satisfied andTI reveals the entire set of faultsB. In this case,TI is a thorough test satisfying COMPLETE(TI,CI) and represents an ideal test set.

2.2.3 Program Errors

Any approach to testing is based on assumptions about the way program faults occur. Faults are due to two main reasons:

• Faults occur due to our inadequate understanding of all conditions with which a program must deal.

• Faults occur due to our failure to realize that certain combinations of con- ditions require special treatments.

Goodenough and Gerhart classify program faults as follows:

• Logic Fault: This class of faults means a program produces incorrect results independent of resources required. That is, the program fails because of the faults present in the program and not because of a lack of resources. Logic faults can be further split into three categories:

Requirements fault: This means our failure to capture the real require- ments of the customer.

2.2 THEORY OF GOODENOUGH AND GERHART 35