Test Patterns – Please Stand By

John D. McGregor


I guess I will date myself somewhat but I can remember getting up to watch Saturday cartoons, only to hear a steady hum and see the picture shown in Figure 1. This "test pattern" was broadcast every time the station left the air even for a moment and then returned. It was intended as a standard against which cameras and television sets could be calibrated. This month I want to discuss a technique that I have been using for sometime to improve the quality and efficiency of the testing process. It borrows from the popular "design pattern" technique to provide a means to share test construction experience. I have found it doubly productive to couple my test pattern development with the use of design patterns. I am slowly recognizing and capturing the patterns of tests that correspond to specific design patterns.

Figure 1


Design patterns describe interactions between classes and determine the specification of the classes that participate in the pattern. The test pattern that accompanies a design pattern defines a configuration of objects needed to test the interactions between classes that are integrated according to the design described by the pattern. This is the level of testing that I have referred to previously as cluster testing. The cluster test suite covers the set of behaviors contributed to the pattern by each of the participating objects. A class may participate in several different clusters defined by several different patterns. The union of the roles for this class is the basis for the class test suite.


The goal of a pattern-based design process is a high quality application with a well-considered structure.

Patterns have been used to reproduce pieces of successful designs for many years. Henry Ford mass produced affordable automobiles by creating each part from a pattern that guaranteed it would fit in the space provided and that it would interact correctly with its neighbors. My mother benefited from the design expertise of someone she never met by using a McCall’s pattern to sew a dress. These physical patterns captured and communicated certain information about the individual pieces that contributed to the design. The user of the pattern had to provide an essential creative element, an overall vision of the whole product.

A software design pattern is a statement about the whole with some information about the parts needed to achieve that whole. A design pattern is much more mutable than those used to manufacture automobiles or to sew clothing. A software design pattern does not prescribe an exact solution that can be applied precisely. Christopher Alexander, the architect who is an original source of the patterns approach, expected that a pattern could be applied a million times and not be used in exactly the same way twice [1].

Good programmers take advantage of patterns wherever they apply. A pattern description "qualifies" where a pattern is applicable. Too often an overeager developer will stretch the applicability of a pattern resulting in a less than optimal result. At the same time, a variation in circumstances should result in a variation in which pattern is applied.

Reuse via patterns seldom is verbatim reuse. The pattern provides high level guidance rather than detailed implementation instructions. The central point of this article is that I can save testing time and effort by recognizing that the test of a segment of code that was designed using a design pattern actually follows a pattern itself. That test pattern can be reused wherever similar designs are used; however, each instance of use will be tailored to the exact needs of the current situation.

A Pattern and its Description

A pattern solves a reoccurring problem by defining a software configuration that is the best solution within the described context as decided by examining the forces that mitigate for one solution or another. The pattern is the best solution possible given the forces influencing the system at the point where the pattern is applied. The description of the pattern is an important part of the pattern. The pattern description explains the pattern to the reader so that they do not have to discover it anew. In particular, the description provides a complete explanation of the problem including the context within which the problem is sited and the forces that influence which alternative is the best. The description provides an analysis of the alternative designs and communicates the optimal solution without prescribing an exact implementation.

What’s Different?

Test patterns are design patterns. Both are intended to guide the construction of a piece of software. In both cases we want the software to be well designed. What’s different is the intent of the software.

Consider the difference in intent between a developer and a tester. The developer intends to compute an answer given an input and an algorithm. At the same time the developer must handle a wide range of possible input values and must prepare to continue computing regardless of input. The tester’s intent is to handle very specific inputs chosen to probe specific application functionality. Test code is usually simpler than the application code it tests and needs a simpler design. However, the test software often requires equally sophisticated techniques. For example, in our implementation of the PACT architecture for Java [5], we use reflective programming techniques.

Pattern Descriptions

There are several widely used formats for pattern descriptions. Analysis pattern descriptions [3] tend to have a few different fields from the description for a "Gang-of-Four" (GOF) design pattern [4]. Although I have a format that I favor, it is clear that a test pattern should adopt the format used by its "parent" design pattern. The fields in a test pattern have slightly different content from the design pattern. In Table 1 I have listed each section from the format used by the GOF and then I give a brief description of the content of the same field in a test pattern.

Table 1



GOF Design Pattern Usage

Test Pattern Usage


Short description of the solution

Describe the aspects of the design that will be tested


A description of the problem being solved and how the solution solves it

A description of the design pattern and why this test pattern is a good way to test it


How to know when to use this pattern

Only used if the test pattern is not always used for all instances of the design pattern


A UML class diagram that shows the basic relationships between classes.

Specification of the test classes (including the specific test cases) and its interactions with the Cluster Under Test (CUT)


What is different about the overall finished design because of the use of this pattern.

Describes how well covered the CUT will be as a result of using this pattern.


A description of basic implementation decisions that must be made.

Same – includes gaining access to the attributes and behaviors in the production software


A description of the interactions between classes that participate in the pattern.

Particularly used to describe interactions between the CUT and the test software.

Sample Code

Segments of code to illustrate a specific use of the pattern

Provides details of how to integrate into the PACT.

Known Uses

Examples of where the pattern is used

Not used because few people publish their test software!

Related Patterns

Other patterns that may be useful for determining the details of this pattern.

Other patterns, design and test, that are useful for implementing this pattern.


An Example – Observer

The Observer pattern has been extensively used. I want to use this design pattern and its related test pattern, the ObserverTest pattern, to illustrate the concept. So first I will summarize the Observer pattern description. A more complete description can be found in [4].


The intent of the Observer pattern, shown in Figure 2, is to maintain a dependency between a central object (Subject) and multiple dependent objects (Observers). When the Subject changes state, the dependent objects are notified so that they may update themselves. The update is based on the new state of the central object. Each Observer queries the Subject for the information needed by that specific Observer.

Figure 2 – The Observer Pattern







The Subject and Observer objects are defined separately from each other. New Observer types may be created without modification of the Subject. Additionally,


Implementation Issues

The discussion in [4] identifies several implementation issues. These issues are a source of test cases that can be applied to any cluster designed using the pattern..


Example Usage

The JavaBean component model supports a pattern that I will refer to as VetoableChange. The pattern is a variant of the Observer pattern. The intent of this pattern is to maintain a constraint in which the Observer objects can veto a change that the Subject proposes to make. A class can implement one of the Listener interfaces (e.g., VetoableChangeListener) and then its instances become eligible to register with Subjects that fire VetoableChanges. The Listener objects are intermediary objects, called Change Managers in [4]. These objects are created and added to the Subject by Observer objects. When the Subject changes state, the VetoableChange message is sent to all of the listening objects. The Listener sends a message, specified by overriding the constructor of the Listener, to the Observer. The Observer may then query the Subject for information. If the object decides to veto the change, it fires a PropertyChangeException and the change is not made. If it does not intend to veto the change, it does not send any message. Eventually the Subject sends a PropertyChange message that is received by a PropertyChangeListener instead of the VetoableChangeListener.

Figures 3 and 4 illustrate this application of the Observer pattern. In Figure 3, the proposed change is accepted and in Figure 4 it is rejected.

Figure 3 – An accepted change

Figure 4 – A rejected change


The ObserverTest Pattern

A portion of the design of an application has been created using the Observer pattern. Our intent is to design effective and efficient interaction test software that tests

An ObserverTest object will "observe" the Subject and the Observer to determine whether each test case passes or fails. The ObserverTest pattern is an extension of the PACT generative pattern language [5].

There is a need to test the interactions between the Subject and Observer objects. Both the Subject and Observer implementations have been tested in isolation; however, the interaction between the two has not been tested and can be fairly complex. The Subject may not send notifications to its Observers when it should. The Observer may not send the correct queries of the Subject.

The tests will be conducted within the context of an application whose design is the composition of the Observer pattern and several other design patterns. Some of the Subject and Observer objects may play roles in multiple design patterns. This implies that the ObserverTest pattern may only engage a portion of the interface of each class. The test code does not require any modification of either the Subject or the Observers and can focus on the behaviors and state of the objects that are specific to the Observer pattern.

There are several forces that constrain the design of the test software and the specification of the ObserverTest class.

The structure of this pattern is shown in Figure 5. I will Build a Test Class for each Significant Application Class as defined in the PACT pattern language [5]. Define an ObserverTest class for each Subject/Observer pair of classes. Create an ObserverTest object for each ConcreteSubject/ ConcreteObserver association. Register each test object with the ConcreteSubject object as an Observer. Each test object contains a reference to an Observer object. The ObserverTest object invokes the interaction between objects and accesses the state of both objects to determine whether the state of each is correct. The test software will Validate the Test Results Within the Test Case [5] whenever possible.

Include the interface of the Observer interface in the interface of the ObserverTest class. Inherit from the PACT abstract classes. Specify the test suite, test script and test case methods needed to test the interaction between the Subject and Observer. Select test cases based on specific types of test cases.

At the most abstract level there is only one type of test case. In the test case, an ObserverTest object invokes a Subject method that is intended to invoke a state change and a notification. When the ObserverTest object receives notification of the change from the Subject it will check that (1) the appropriate notification is being sent and (2) its associated Observer object has completed the intended action and is now consistent with the Subject. This type of test case produces an actual test case for each state change that is intended to lead to a notification.





Figure 5 – The ObserverTest Pattern


The ObserverTest object uses the standard interfaces of the Subject for their interaction. Use the addObserver interface to setup the test environment. Specialize the ObserverTest class to each Observer/Subject pair so that it knows which methods in the query interface of the Subject object to use. The specialization is required to represent the unique portion of the state of the Subject that would be requested by the Observer. The ObserverTest object must know the states in which the notification should place the Observer and what messages it was designed to send to the Subject.

The ObserverTest object initiates the test sequences in response to one of its test suites being invoked. It creates the Subject and Observer objects, sets up the dependency between specific Subject/Observer objects and then invokes methods on the interface of the Subject in order to induce a notification. The sequence of events is shown in Figure 6.


Figure 6 – ObserverTest implementation


The main consequence of applying the ObserverTest pattern is the ability to achieve an "all notifications" level of test coverage. That is, every possible notification is sent in all possible situations. This increases our confidence that the interactions have been properly designed and implemented. Since the Observer pattern is an extensibility device, we also have reason to believe the system is extensible.

The ObserverTest pattern is related to the PACT pattern language. The ObserverTest class is implemented as a PACT class. That implies it encapsulates the test cases needed to test the interactions between the Subject and Observer as well as the logic needed to verify the state of the Subject and Observer objects.

As an example, consider applying the canonical test case for the Observer test pattern to code that follows the VetoableChange pattern. The TestObserver adds a VetoableChangeListener for itself with the Subject object. The TestObserver adds a VetoableChangeListener for the Observer object to the Subject. The TestObserver also adds corresponding PropertyChangeListeners. When the TestObserver receives the notification, it checks the state of the Subject and Observer objects. The two objects should be consistent, but in the original state. After the PropertyChange signal is fired, the Subject and Observer objects should be consistent, but in the new state.

Since the VetoableChange pattern is more specific than the Observer pattern, additional test cases can be defined. In particular, a second type of test case can be defined which fires a VetoableChange but results in a PropertyChangeException from the Observer rather than a PropertyChange from the Subject, shown in Figure 6. In this case, the two objects should still be consistent and should still contain the values that were present when the VetoableChange signal was fired. Several actual test cases can be constructed from this type of test case.

Evaluation of the ObserverTest Pattern

Jim Coplien evaluates a pattern on his web page [3] using the criteria that I will use here to evaluate the ObserverTest pattern.


Analysis of the Product Under Test is the most time-consuming, and therefore most costly, part of testing. Recognizing patterns is an important step in analysis and potentially a major time saver. By reading the design documentation and identifying standard design patterns used by the development team, the tester can apply their own patterns. The development of a test pattern requires the same level of abstraction as is found in the design pattern. The test pattern addresses the fundamental nature of the design pattern while permitting many different implementations. The code, which tests an application of the Observer pattern, will vary greatly from one instance to another.

I have shown only one of a number of test patterns that many other testers, and I have used for some time. In future columns I will provide additional patterns and I welcome contributions from others in the testing community.


1. Christopher Alexander. The Timeless Way of Building, Oxford University Press, 1979.

2. James Coplien, http://hillside.net/patterns/definition.html.

3. Martin Fowler. Analysis Patterns: Reusable Object Models. Addison-Wesley, 1996.

4. Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides. Design Patterns: Elements of Reusable Object-oriented Software, Addison Wesley, 1994.

5. John D. McGregor, Parallel Architecture for Component Testing, Journal of Object-Oriented Programming, May 1997.