Planificación basada en casos

the control structures in the code and adds probes (lines of code that say that the program has reached ; certain place). It adds a probe for each branch in the program. Inserting these probes into the code is callei

instrumenting the program. You know that you've pushed the program down every branch when all probe

have been printed.

Along with instrumenting the code, a good coverage monitor captures probe outputs, analyzes, an< summarizes them. It counts the different probe messages, reports the percentage of probes triggered so fa and so reports on thoroughness of testing done so far. It might also report untriggered branches. Create test: to exercise those parts of the program.

Coverage monitors are designed for glass box testing, but you can use one without knowing the internals of the program under test. It helps to have a listing of the program, but even without one, you can find out your level of coverage of the program, which can be important feedback.

Many coverage monitors are available commercially. To encourage the programming staff to use one, build a good file on what's available. However, the programmers may raise honest and considerable objections:

• These monitors insert code into the program: The shipping program won't

include this test code. It is risky to test a program that significantly differs in its control flow (as these do, because they pass through the monitor routines all the time) from the one you will release. • The monitoring software makes the program run more slowly: It might create race conditions that

can't arise in real use of the final product, and it might hide race conditions and other performance problems that will arise in the real product.

• The instrumented program is much larger than the final product: It might very well not fit on test machines of interest, and its size might disguise the fact that the non-instrumented program has also grown to be too big.

• Some of this software is full of bugs: Programmers get upset when they spend two days tracking down a bug in the program only to find that it's just another bug in the test tool. Before buying and using a coverage monitor, ask the monitor's developer for a list of its known bugs. Since this program is designed to help find bugs, a request for the product's known bugs is not unreasonable.

A

The programmer often knows that something must be true at a certain point in the program. At this point, many programmers will test an assertion that this something is in fact true. This is an assertion check. The typical program does nothing visible if the assertion is correct, but prints or displays an error message if the assertion is false. Other programs print nothing, but silently execute error recovery code.

To make the internal testing visible in the program, have the programmer log a message to the screen or printer whenever an assertion (or a particularly interesting assertion) is tested. Assertion checking is most often made visible to the tester to help track down bugs that are hard to reproduce or that the programmer doesn't understand how to solve. To keep the noise level down, usually the only assertions made visible are those in the suspected area of the program.

167

M

The running program uses memory in the following ways:

• Some memory is used by the code itself: Except for (very unfashionable) self-modifying programs, no programs can write to the code area.

• Some memory is used for data: The program can read from and write to this area of memory, but it shouldn't point the CPU to execute its next instruction from a data area.

• Some memory is hardware related: The program can talk to an external device by reading from or writing to these memory addresses. However, many programs talk to these areas indirectly, relying on the BIOS (basic input/output system) or even higher-level device handlers supplied with the system to talk to the hardware. In these cases, the program is behaving incorrectly (probably wildly) if it accesses a hardware address.

• Some memory is out of bounds: The operating system allocates a certain memory area to each program. In multi-tasking systems, the program is forbidden from accessing most of the computer's memory. In all systems, the program can't access memory that the computer doesn't have. A memory-usage checking program will report suspicious memory accesses. Depending on the program and the options you use, it might stop and dump status data, it might jump into a debugger mode, or it might just flag the event and move on.

A different type of memory report that testers often find useful states how much free memory is available and reports the size of the largest block (or few blocks) of free memory. It's also handy to (optionally) get a more detailed listing for all of memory (or all memory in use by the program). This listing shows what areas of memory are free and what routines are using the rest, block by block. This report is handy because memory usage problems often don't result in visible misbehavior until long after they've occurred. You can use this tool to discover:

• How much memory each feature or option takes.

• Whether the program cleans up after a given graphic, feature, or dialog (frees up the memory it was using) when you are done with it. If not, the program will eventually run out of memory.

This tool often presents a cleaner picture because it has to do some cleanup in the process of getting the data for the report. Sometimes, using the tool will block you from reproducing the bug. However, that information is often useful and significant to the programmer in its own right.

In our experience, the memory usage report is the single most valuable tool the programmer can build into the code for the benefit of the tester.

168