Inspections, Walkthroughs & Reviews

Summary

Introduction

The concept of studying program code as part of program testing is very widely accepted. For this reason this chapter will concentrate on human-testing (non-computer based testing) before discussing in depth computer- based testing techniques (Test Case Design and Unit Testing).

Human testing techniques can be quite effective in finding errors - so much so that every serious programming project should use one or more of these techniques. Human-testing techniques should be applied between the time the program is coded and the time when computer-based testing begins.

Inspections and Walkthroughs

As the title suggests, the two primary human testing methods are code inspections and walkthroughs. Because these two human-testing methods have lots in common, this section will discuss their similarities. Their differences are discussed in separate sections.

Inspections and Walkthroughs involve a team of people reading or visually inspecting a program. With either method, participants must do some preparatory work following by the all-important "meeting of the minds". The objective of this meeting is to find errors but not to debug them. In other words, to test.

In inspections/walkthroughs, a group of developers - usually 3 to 4 - performs the review. Only one of the participants is the author of the code. Therefore, the majority of the testing is conducted by people rather than by the author. The main advantages of inspections and walkthroughs are:

Of course, inspections and walkthroughs are not flawless, but it has been found that inspections and walkthroughs do complement computer-based testing. If fact, error-detection efficiency suffers if one or the other is not used.

Code Inspections

Code inspection is a set of procedures and error-detection techniques for group code-reading. Code inspections work as follows:

An inspection team usually consists of 4 people:

  1. A moderator.

  2. The programmer.

  3. The program's designer (if different from the programmer).

  4. A testing specialist.

The moderator must be a competent programmer but should not be the author of the code and need not be acquainted with the details of the code. The duties of the moderator are:

The moderator distributes the location of the code and the design specification to the other participants several days in advance of the inspection sessions. The participants are expected to familiarize themselves with the material prior to the session (but nobody does!) During the session two activities occur:

  1. The programmer narrates, statement by statement, the logic of the program. During the narration, other participants may raise questions and these questions should be answered fully to determine if errors exist. It is likely that the programmer rather than the other team members will find many of the errors. For example, in a recent code inspection session I was asked to explain how a thread-chained work units module works. The design of this module was required to run each work unit in a separate thread, and if the number of active threads exceeded a configurable limit, work units should be placed in a waiting queue. While explaining the logic of placing work units in a waiting queue, it occurred to me that a certain method did not properly pop items from the waiting queue once the thread count dropped below the maximum threshold.

  2. The program is checked with respect to a checklist (see below) of historically common programming errors.

Time and location of the inspection should be planned so as to avoid all outside interruptions. The optimal amount of time seems to be 90 to 120 minutes. Obviously, large programmes should be done in multiple inspections, with each inspection dealing with one or several modules.

Also note that the right attitude should be established in order for the inspection process to be effective. If the programmer views the inspection process as an attack on his style/character, then the process will be quite ineffective. Rather, the programmer must approach this process with an egoless attitude and must consider this inspection as a way to highlight possible weaknesses and receive feedback concerning programming style, choice of algorithms and programming techniques. Again, the objective of inspections is to find errors. For this reason, it is always recommended that the results of the inspection be shared only between the participants. Managers need only know that 'few errors have been identified and are being fixed', no more, no less.

Finally, note that the inspection process is a way of identifying early the most error prone sections of the program, helping to focus more attention on these sections during computer-based testing.

Inspection Checklist

An important part of the inspection process is the use of checklists to examine the program for common errors. Coming up with your own checklist can be a very time consuming and open-ended project. I highly recommend using FxCop, a simple but powerful and useful tool that ensures assemblies are written according to Microsoft .NET Framework Design Guidelines. FxCop has many checklists with hundreds of rules. There are checklists for design, globalization, interoperability, naming, security, usage, etc.

The following shows a snapshot of FxCop after an assembly has been inspected according to FxCop rules:

Walkthroughs

Like code inspections, code walkthroughs is a set of procedures and error-detection techniques for group code reading. Whilst very similar to inspections, walkthroughs uses a different set of procedures and error detection techniques.

Like a code inspection, a code walkthrough is an uninterrupted meeting of 1- 2 hours in duration. The walkthrough team consists of 3 - 5 people. One of these people plays a role similar to that of the moderator in code inspections, another person takes notes (errors) down, and a third person plays the role of the tester. Suggestions vary as to who the 3 - 5 people should be. Of course, the programmer should be one of these people. Suggestions for the others include 1) a highly experienced programmer, 2) a programming language expert, 3) a new programmer to give a fresh unbiased feedback, 4) the person who will eventually maintain the program, 5) someone from the same team as the programmer, and so on.

The initial procedure is identical to that of the inspection process. Participants are given the material several days in advance to review (of course, very few do). During the meeting, rather than simply read the program and go through checklists of common errors, the participants play computer. The designated tester should have a small set of paper test cases (inputs and expected outputs) for the program. During the meeting, each test case is mentally executed, that is, the test data is walked through the logic of the program. The state of the program (i.e., variable values) is monitored on paper.

Note that the test cases themselves do not play a critical role, rather, they serve as a vehicle for getting started and for questioning the programmer about his/her logic and assumptions. In most walkthroughs, more errors are found during the process of questioning the programmer than are found directly by the test cases themselves.

As in inspections, attitudes should be properly checked. Comments should be directed towards the program rather than toward the programmer. And similar to inspections, the walkthroughs should have a follow-up process. Also, side effects observed from inspections (identification of error-prone sections, and education in errors, style and techniques) also apply to the walkthrough process.