I guess I will date myself somewhat but I can remember getting up to watch Saturday cartoons, only to hear a steady hum and see the picture shown in Figure 1. This "test pattern" was broadcast every time the station left the air even for a moment and then returned. It was intended as a standard against which cameras and television sets could be calibrated. This month I want to discuss a technique that I have been using for sometime to improve the quality and efficiency of the testing process. It borrows from the popular "design pattern" technique to provide a means to share test construction experience. I have found it doubly productive to couple my test pattern development with the use of design patterns. I am slowly recognizing and capturing the patterns of tests that correspond to specific design patterns.

Design patterns describe interactions between classes and determine the specification of the classes that participate in the pattern. The test pattern that accompanies a design pattern defines a configuration of objects needed to test the interactions between classes that are integrated according to the design described by the pattern. This is the level of testing that I have referred to previously as cluster testing. The cluster test suite covers the set of behaviors contributed to the pattern by each of the participating objects. A class may participate in several different clusters defined by several different patterns. The union of the roles for this class is the basis for the class test suite.

The goal of a pattern-based design process is a high quality application with a well-considered structure.

Patterns have been used to reproduce pieces of successful designs for many years. Henry Ford mass produced affordable automobiles by creating each part from a pattern that guaranteed it would fit in the space provided and that it would interact correctly with its neighbors. My mother benefited from the design expertise of someone she never met by using a McCall’s pattern to sew a dress. These physical patterns captured and communicated certain information about the individual pieces that contributed to the design. The user of the pattern had to provide an essential creative element, an overall vision of the whole product.

A software design pattern is a statement about the whole with some information about the parts needed to achieve that whole. A design pattern is much more mutable than those used to manufacture automobiles or to sew clothing. A software design pattern does not prescribe an exact solution that can be applied precisely. Christopher Alexander, the architect who is an original source of the patterns approach, expected that a pattern could be applied a million times and not be used in exactly the same way twice [1].

Good programmers take advantage of patterns wherever they apply. A pattern description "qualifies" where a pattern is applicable. Too often an overeager developer will stretch the applicability of a pattern resulting in a less than optimal result. At the same time, a variation in circumstances should result in a variation in which pattern is applied.

Reuse via patterns seldom is verbatim reuse. The pattern provides high level guidance rather than detailed implementation instructions. The central point of this article is that I can save testing time and effort by recognizing that the test of a segment of code that was designed using a design pattern actually follows a pattern itself. That test pattern can be reused wherever similar designs are used; however, each instance of use will be tailored to the exact needs of the current situation.

A pattern solves a reoccurring problem by defining a software configuration that is the best solution within the described context as decided by examining the forces that mitigate for one solution or another. The pattern is the best solution possible given the forces influencing the system at the point where the pattern is applied. The description of the pattern is an important part of the pattern. The pattern description explains the pattern to the reader so that they do not have to discover it anew. In particular, the description provides a complete explanation of the problem including the context within which the problem is sited and the forces that influence which alternative is the best. The description provides an analysis of the alternative designs and communicates the optimal solution without prescribing an exact implementation.

Test patterns are design patterns. Both are intended to guide the construction of a piece of software. In both cases we want the software to be well designed. What’s different is the intent of the software.

Consider the difference in intent between a developer and a tester. The developer intends to compute an answer given an input and an algorithm. At the same time the developer must handle a wide range of possible input values and must prepare to continue computing regardless of input. The tester’s intent is to handle very specific inputs chosen to probe specific application functionality. Test code is usually simpler than the application code it tests and needs a simpler design. However, the test software often requires equally sophisticated techniques. For example, in our implementation of the PACT architecture for Java [5], we use reflective programming techniques.

There are several widely used formats for pattern descriptions. Analysis pattern descriptions [3] tend to have a few different fields from the description for a "Gang-of-Four" (GOF) design pattern [4]. Although I have a format that I favor, it is clear that a test pattern should adopt the format used by its "parent" design pattern. The fields in a test pattern have slightly different content from the design pattern. In Table 1 I have listed each section from the format used by the GOF and then I give a brief description of the content of the same field in a test pattern.

The Observer pattern has been extensively used. I want to use this design pattern and its related test pattern, the ObserverTest pattern, to illustrate the concept. So first I will summarize the Observer pattern description. A more complete description can be found in [4].

The intent of the Observer pattern, shown in Figure 2, is to maintain a dependency between a central object (Subject) and multiple dependent objects (Observers). When the Subject changes state, the dependent objects are notified so that they may update themselves. The update is based on the new state of the central object. Each Observer queries the Subject for the information needed by that specific Observer.

The Subject and Observer objects are defined separately from each other. New Observer types may be created without modification of the Subject. Additionally,

• Abstract coupling between Subject and Observer – Details of the coupling do not need to be repeated for each concrete instantiation
• Support for broadcast communication – The ConcreteSubject notifies all of its ConcreteObservers. This can be a large number of objects and a large amount of messaging when in fact only a few of the Observers care about the specific change that has occurred.
• Unexpected updates – Indirect couplings between Subjects and Observers may cause a cascade of updates.

The discussion in [4] identifies several implementation issues. These issues are a source of test cases that can be applied to any cluster designed using the pattern..

• How are observers stored in the Subject?
• Can an Observer observe more than one subject?
• Who calls notify? Each state change operation?
• How are Observers notified that a Subject is being deleted?
• When is the Subject sufficiently stable to call notify?
• What information should notify pass to the Observers?
• Must every Observer be notified of every change?
• Should other objects, such as a Change Manager, be used to encapsulate complex update logic?

The JavaBean component model supports a pattern that I will refer to as VetoableChange. The pattern is a variant of the Observer pattern. The intent of this pattern is to maintain a constraint in which the Observer objects can veto a change that the Subject proposes to make. A class can implement one of the Listener interfaces (e.g., VetoableChangeListener) and then its instances become eligible to register with Subjects that fire VetoableChanges. The Listener objects are intermediary objects, called Change Managers in [4]. These objects are created and added to the Subject by Observer objects. When the Subject changes state, the VetoableChange message is sent to all of the listening objects. The Listener sends a message, specified by overriding the constructor of the Listener, to the Observer. The Observer may then query the Subject for information. If the object decides to veto the change, it fires a PropertyChangeException and the change is not made. If it does not intend to veto the change, it does not send any message. Eventually the Subject sends a PropertyChange message that is received by a PropertyChangeListener instead of the VetoableChangeListener.

Figures 3 and 4 illustrate this application of the Observer pattern. In Figure 3, the proposed change is accepted and in Figure 4 it is rejected.

A portion of the design of an application has been created using the Observer pattern. Our intent is to design effective and efficient interaction test software that tests

• the completeness of the notifications sent out by the Subject to the Observer,
• the correctness of the queries sent by the Observers, and
• the consistency of the data in the Subject and in the Observers.

An ObserverTest object will "observe" the Subject and the Observer to determine whether each test case passes or fails. The ObserverTest pattern is an extension of the PACT generative pattern language [5].

There is a need to test the interactions between the Subject and Observer objects. Both the Subject and Observer implementations have been tested in isolation; however, the interaction between the two has not been tested and can be fairly complex. The Subject may not send notifications to its Observers when it should. The Observer may not send the correct queries of the Subject.

The tests will be conducted within the context of an application whose design is the composition of the Observer pattern and several other design patterns. Some of the Subject and Observer objects may play roles in multiple design patterns. This implies that the ObserverTest pattern may only engage a portion of the interface of each class. The test code does not require any modification of either the Subject or the Observers and can focus on the behaviors and state of the objects that are specific to the Observer pattern.

There are several forces that constrain the design of the test software and the specification of the ObserverTest class.

• Modifying the production software to test it requires additional tests after the production software is returned to its original form. Therefore, we prefer not to modify the software.
• The ObserverTest object must know when the Subject object sends out notifications. The most efficient way to achieve this is to use the test object as an Observer object. Adding the ObserverTest object as an additional Observer to the Subject does not modify the behavior of the production software since it is supposed to be designed to accept an indeterminate number of Observers.
• Having a single test object that manages, accesses, and verifies the state of both the Subject and Observer simplifies the need to verify that the state of the Subject object is properly set.

The structure of this pattern is shown in Figure 5. I will Build a Test Class for each Significant Application Class as defined in the PACT pattern language [5]. Define an ObserverTest class for each Subject/Observer pair of classes. Create an ObserverTest object for each ConcreteSubject/ ConcreteObserver association. Register each test object with the ConcreteSubject object as an Observer. Each test object contains a reference to an Observer object. The ObserverTest object invokes the interaction between objects and accesses the state of both objects to determine whether the state of each is correct. The test software will Validate the Test Results Within the Test Case [5] whenever possible.

Include the interface of the Observer interface in the interface of the ObserverTest class. Inherit from the PACT abstract classes. Specify the test suite, test script and test case methods needed to test the interaction between the Subject and Observer. Select test cases based on specific types of test cases.

At the most abstract level there is only one type of test case. In the test case, an ObserverTest object invokes a Subject method that is intended to invoke a state change and a notification. When the ObserverTest object receives notification of the change from the Subject it will check that (1) the appropriate notification is being sent and (2) its associated Observer object has completed the intended action and is now consistent with the Subject. This type of test case produces an actual test case for each state change that is intended to lead to a notification.

The ObserverTest object uses the standard interfaces of the Subject for their interaction. Use the addObserver interface to setup the test environment. Specialize the ObserverTest class to each Observer/Subject pair so that it knows which methods in the query interface of the Subject object to use. The specialization is required to represent the unique portion of the state of the Subject that would be requested by the Observer. The ObserverTest object must know the states in which the notification should place the Observer and what messages it was designed to send to the Subject.

The ObserverTest object initiates the test sequences in response to one of its test suites being invoked. It creates the Subject and Observer objects, sets up the dependency between specific Subject/Observer objects and then invokes methods on the interface of the Subject in order to induce a notification. The sequence of events is shown in Figure 6.

The main consequence of applying the ObserverTest pattern is the ability to achieve an "all notifications" level of test coverage. That is, every possible notification is sent in all possible situations. This increases our confidence that the interactions have been properly designed and implemented. Since the Observer pattern is an extensibility device, we also have reason to believe the system is extensible.

The ObserverTest pattern is related to the PACT pattern language. The ObserverTest class is implemented as a PACT class. That implies it encapsulates the test cases needed to test the interactions between the Subject and Observer as well as the logic needed to verify the state of the Subject and Observer objects.

As an example, consider applying the canonical test case for the Observer test pattern to code that follows the VetoableChange pattern. The TestObserver adds a VetoableChangeListener for itself with the Subject object. The TestObserver adds a VetoableChangeListener for the Observer object to the Subject. The TestObserver also adds corresponding PropertyChangeListeners. When the TestObserver receives the notification, it checks the state of the Subject and Observer objects. The two objects should be consistent, but in the original state. After the PropertyChange signal is fired, the Subject and Observer objects should be consistent, but in the new state.

Since the VetoableChange pattern is more specific than the Observer pattern, additional test cases can be defined. In particular, a second type of test case can be defined which fires a VetoableChange but results in a PropertyChangeException from the Observer rather than a PropertyChange from the Subject, shown in Figure 6. In this case, the two objects should still be consistent and should still contain the values that were present when the VetoableChange signal was fired. Several actual test cases can be constructed from this type of test case.

Jim Coplien evaluates a pattern on his web page [3] using the criteria that I will use here to evaluate the ObserverTest pattern.

• This pattern solves a real problem. Interactions are a source of many errors in component-based software design. Writing effective test software that examines the interactions is difficult. The ObserverTest pattern provides an effective technique for constructing these tests.
• The design is a proven concept because it is a variant of the Observer pattern. The test software needs to be notified when an action has happened. The ObserverTest pattern allows the software using the Observer pattern to be tested without being modified.
• This solution wasn’t obvious (at least to me). I have seen other approaches to this problem used in the field. However, after careful analysis of the forces, this pattern is the optimal solution.
• The design describes a relationship between the application software and the test software. The pattern describes a set of interactions. The test software initiates an interaction, the Subject object treats the ObserverTest object like any other observer, and the ObserverTest object then accesses attributes of the Subject.
• The pattern has a significant human component in that it appeals to the inherent search for symmetry. (Incidentally, a variation of the ObserverTest pattern uses multiple observers, one in each process.)

Analysis of the Product Under Test is the most time-consuming, and therefore most costly, part of testing. Recognizing patterns is an important step in analysis and potentially a major time saver. By reading the design documentation and identifying standard design patterns used by the development team, the tester can apply their own patterns. The development of a test pattern requires the same level of abstraction as is found in the design pattern. The test pattern addresses the fundamental nature of the design pattern while permitting many different implementations. The code, which tests an application of the Observer pattern, will vary greatly from one instance to another.

I have shown only one of a number of test patterns that many other testers, and I have used for some time. In future columns I will provide additional patterns and I welcome contributions from others in the testing community.

1. Christopher Alexander. The Timeless Way of Building, Oxford University Press, 1979.

2. James Coplien, http://hillside.net/patterns/definition.html.

3. Martin Fowler. Analysis Patterns: Reusable Object Models. Addison-Wesley, 1996.

4. Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides. Design Patterns: Elements of Reusable Object-oriented Software, Addison Wesley, 1994.

5. John D. McGregor, Parallel Architecture for Component Testing, Journal of Object-Oriented Programming, May 1997.