Validating Domain Models

John D. McGregor

I am seldom faced with the development of a single standalone application. It seems that these days everyone is building a suite of inter-related applications, a framework for rapid customization of a basic application or they are developing the requirements for the application as they build it. These types of development rely on well-formed abstractions to be successful.

The best way I know of to produce high-quality abstractions is through domain analysis [2]. This level of analysis seeks to identify the abstractions that describe a specific body of knowledge and the relationships between those abstractions. The resulting model describes a design space larger than that required for a single, specific application. It is supposed to be sufficiently large to include logically related concepts but not so large that valuable time is wasted in intellectual exercises.

One of the most difficult tasks in object-oriented development is the identification of classes. Domain analysis provides a technique for "priming the pump". The basic vocabulary of a domain provides an easily accessible supply of concepts that become classes in the analysis model. Once the domain classes are identified, additional classes are created to describe the interfaces of the domain model with supporting technologies such as the user interfaces and database. Other classes also naturally arise as the domain model is elaborated to describe a specific application.

Recently I found myself involved in a domain modeling effort that, as usual, had to quickly produce high-quality results that added value to the development effort. As the team moved forward identifying classes, I began to think about how we should test the model. This month I want to expand on my previous description of guided inspection [1] and to focus specifically on "testing" the domain model.

Domain Models

Different development methods structure the domain model very differently. At Software Architects we use the complete UML suite of diagram types to represent the knowledge needed to describe the domain. This includes defining a set of high-level use cases that describe how a typical system in the domain is used. Therefore, the complete set of guided inspection criteria can be used in examining the domain model.

A basic premise for domain modeling is that the model is inclusive. Any concept that would be used in any application written within the domain is included in the model. This is also true for the use case model. Any actor for any application would be included in the use case model. This gives new meaning to the "completeness" criteria of guided inspection.

Domain models are abstract. Concepts are represented as generally as possible. Constraints are stated as liberally as applicable. While this produces a model that encompasses as many applications as possible, it also makes it difficult to evaluate the "correctness" of a use of an application.

A domain model may be inconsistent. By being inclusive of all potential applications, the model may contain concepts that would always appear separately in an actual application. The use of the inheritance notation in UML supports the inclusion of alternatives. A car contains a single transmission but the domain model should present all possible types of transmissions that are used.

Using Guided Inspection

Guided inspection is an inspection technique that is "guided" by test cases. Inspections are used to provide a detailed examination of a design or program by a human, as opposed to a machine’s execution of a prototype or completed application. This inspection is often a top down reading of the code or a scan of a diagram. Techniques such as checklists have been used to summarize the results of an inspection and to ensure that the inspector does a thorough job. The top down approach makes the measurement of coverage straightforward but it is more difficult for the inspector to ensure that appropriate connections have been made between objects.

Objectivity

For testing to be effective it must be conducted objectively. A person testing their own code is seldom sufficiently objective to achieve optimum results. For our domain analysis, objectivity is even more critical than usual because at this level there often are no absolute answers. Two domain experts may differ significantly in their response to the same question. Therefore:

Guideline # 1 – Use a different domain expert during the validation of the domain model from the expert(s) used during modeling.

One problem with this guideline is that each domain expert has his or her own perspective on the domain. For example, one expert is associated with the use of the application with clients while a second expert may be a software developer. Be ready to negotiate between the experts!

Traceability

For testing materials to be maintainable it must be easy to map changes in the domain model to needed changes in the test scenarios. In an iterative development environment changes occur frequently to all parts of the project. Changes in requirements are reflected in changes to the use cases. Changes in class specifications should signal the need for regression testing of the effected parts of the model.

Guideline # 2 – Maintain a mapping between use cases and the classes/packages that realize those use cases.

The obvious problem with this strategy is the potentially large amount of data that can be generated. A spreadsheet application handles this quite well as does a personal database. The positive aspect is that this information s useful for a variety of purposes including assigning use cases to releases and scheduling class development in response to the release schedule. Figure 1 illustrates the matrix.

Figure 1

 

Testability

For testing to be possible the model must be testable. This implies that the model is sufficiently specific to support the evaluation of results. Domain models are general by design and there is a fine line between vague generality and sufficient detail to support testing.

Guideline # 3 – Assign a team member to write test cases as the modeling proceeds. Have the "testing" domain expert review these. Feedback, into the modeling process, information about places at which the model is too vague.

This is common advice that I give to process definers at all levels. There should be a validation activity for each development phase. Preparation for that activity should proceed in parallel with the development activity. This allows the act of preparation to actually help before the formal validation. Writing test cases is an excellent source of continuous feedback during development.

Coverage

For testing to provide us with confidence we need to know how thoroughly the product under test has been examined. The general term for this type of metric is coverage. When we speak of "functional" testing, we mean that the coverage will be expressed in terms of the functional specification of the product under test. The metric is chosen to give some notion of completeness at the appropriate level.

For guided inspection there are two different possible bases for coverage: the class/state/activity and use case diagrams. The use case diagram is a good source of scenarios; however, we are more concerned that the domain model contains a complete set of concepts for the domain. These are represented in the class diagram and further clarified in the state and activity diagrams for each class.

Guideline # 4 – Use copies of the diagrams and mark off each element in a diagram as it is used in a scenario.

Developing a test scenario for each actor in the use case diagram is a minimal level of coverage. One scenario per primary use case is a stronger coverage criterion. Covering every primary use and then adding coverage for all "alternate courses of action" for use cases that are rated frequent and critical is an even stronger criterion. Once the set of scenarios are run through the model, the resulting coverage of the class diagram and state diagrams provides a check of the thoroughness with which the model has been inspected. Figure 2 illustrates marking the class diagram (not exactly high tech, but it provides a quick visual check).

Figure 2

 

Criteria for a good model

The criteria that I described previously [1] are:

Completeness – The usual defects related to completeness include:

In a domain model there is actually a reverse kind of completeness fault. Domain models may actually include too much information. There is always a tendency to include implementation details in the model particularly when the project is reengineering an existing application. The risk of "over completeness" is that implementation decisions are made too soon. By inclusion in an analysis model these decisions may be viewed as requirements rather than the target of critical examination.

Correctness – Typical correctness defects found in analysis models include:

Domain models are sufficiently general that it can be difficult to determine whether answers to specific questions are correctly answered in the model. No implementations are available to provide specific answers. Basically, an algorithm is correct if it is possible to trace a path through a set of objects that have the appropriate methods and attributes.

Consistency – Consistency defects include:

As I stated earlier, it is acceptable for a domain model to contain contradictory information, so what is "acceptable" as an inconsistency and what is not? The inclusion of alternatives among which there is a clear selection process is acceptable. Accidental inconsistencies that result from failing to clearly resolve an issue are not acceptable.

A Basic Process

Roles

There are essentially two roles in this testing process: modeling expert and domain expert. The domain expert is responsible, naturally, for information concerning the domain. The modeling expert provides expertise in the syntax and semantics of the diagrams. The modeling expert usually is more familiar with the mechanics of the guided inspection process and therefore also serves as the driving force for the testing process. Figure 3 illustrates that even a single modeling expert can usually manage several domain experts working through both the model development and model testing processes.

Figure 3

Steps

The model testing process is tightly coupled with the model development process. We have found it useful to iterate within the modeling process by periodically switching from the modeling activity to the testing activity. This provides quick feedback and often provides new information to be modeled.

The basic steps are the same as for any testing process:

I often perform the role of modeling expert. If the project is sufficiently large for there to be two or more modeling people, I will divide my time and spend as much of my time in the analysis and construction phases of the test process as I can. This allows me to provide almost continuous feedback into the modeling process. I do this by constructing a few basic scenarios, even before the use cases are developed, and then, every time there is a major refactoring of the model, I will "run" the scenarios to determine whether anything has been lost.

 

Techniques

In this section I want to discuss the techniques that we use to examine the domain model.

  1. Test cases – This is the standard technique used in guided inspection to drive the process. Below I will discuss specific techniques for creating test cases from use cases.
  2. Summary Checklist – Both the modeling and domain experts are guided by a checklist that reminds them of qualities to look for in the model. The checklist describes qualities that give an overall evaluation of the model as opposed to the test cases that are passed/failed on an individual basis and that point to defects that are usually local in nature. I will cover this in a future column.

 

Constructing Test Cases

The inspection of the domain model is guided by test cases developed from the use case diagram of the model. Each use case is the source of a set of scenarios that exercise the classes in the model to achieve some use of the system. Replacing the general concepts referenced in the use case description with specific instances creates a scenario. The pre-conditions stated in the use case are translated using the same specific instances. In fact, any constraints, exceptions or other assertions can be translated using the same set of instances.

For example, consider the following use case from our Game framework:

The Player selects an unoccupied space on the gameboard when it is the Player’s turn. The system responds by placing the appropriate token in the Game’s state vector in the location that corresponds to that space.

Replacing the abstract entities with specific instances creates a useable test scenario. For the above use case the following substitutions could be made:

Player

Player0

Unoccupied space

The last space on the right in the top row

Gameboard

Tic Tac Toe grid

Appropriate token

a capital X

The test scenario becomes:

Player0 selects the last space on the right in the top row on the Tic Tac Toe grid using the mouse when it is Player0’s turn. The system responds by having the Player0 object invoke the setMove method on Game which in turn places a capital X in the Game’s state vector in the location that corresponds to the last space on the right in the top row.

Generalization/specialization occurs often in a domain model. We apply specialization to actors as well as classes (this is supported by Rational’s Rose). We can derive scenarios for the specialized actor from the scenarios for its parent. For example, when we specialize the Player to a ComputerPlayer object, the scenario now becomes:

ComputerPlayer0 directly sets its move to be to the last space in the top row on the Tic Tac Toe grid when it is ComputerPlayer0’s turn. The system responds by having the ComputerPlayer0 invoke the setMove method on Game which in turn places a capital X in the Game’s state vector in the location that corresponds to the last space on the right in the top row.

The exceptions section of this use case states:

If the Player selects a non-existent location on the gameboard an exception results.

This is translated into:

If Player0 performs a left mouse button press/release outside the gameboard a PropertyChange exception results.

The main scenario above could be replicated for each space on the gameboard. At the implementation level this would be a valid strategy. At the domain level there is insufficient detail to see any difference between these scenarios so there is no need to expend effort on them. Each scenario should involve a different set of objects not just minor variations.

 

Symbolic Execution

The scenarios are used to drive a symbolic execution. Although there are tools that can automate some or all of this process, their use is still very limited so I will describe the manual process that is used by many projects. I use the term symbolic to characterize the execution because there is no code to execute. In fact, at the domain level, the "glue" classes are missing so a number of assumptions must be made during execution. For example, in the model we will represent a concept, such as Employee, but in the working application there will be a collection of instances of this concept, employees of a specific company. The collection is not modeled since it does not add anything to understanding the domain. During execution, we assume that the instances of that concept are retrieved from the non-existent collection.

The symbolic execution is often recorded on a sequence diagram, such as the one shown in Figure 4. As the scenario is processed, messages are added to the diagram. A couple of items should be checked as each new message is encountered.

Other diagrams should be used and actively marked to record the execution. The state diagram, Figure 5, for the receiving class should be examined to determine whether the instance is in the appropriate state to receive the message. I also mark the initial state and mark as the symbolic execution moves through the states. This helps me later to judge how completely the class has been covered. Finally, Figure 6 shows an activity diagram. I mark one of these to show "path" coverage through complex algorithms. This diagram summarizes several sequence diagrams.

Figure 4

 

Figure 5

Figure 6

Conclusion

I have presented a more detailed view of guided inspection from the perspective of the domain analysis model. The domain model provides a critical foundation for the business objects in the final application. Detecting defects in this very early model will contribute greatly to the quality of the application.

This testing technique is closely coupled to the development process. Even though I have talked of using a different domain expert to drive the test process, all the experts are involved to varying degrees. The symbolic execution session is best accomplished in an interactive session with all of the experts participating.

References

  1. John D. McGregor. The Fifty Foot Look at Analysis and Design Models, Journal of Object-Oriented Programming, July/August 1998.
  2. Reuben Prieto-Diaz. Domain Analysis, Software Engineering Notes, ACM Press, 199x.