CpSc 873

Verification and Validation

Name ____________________________________

There are 5 questions. Answer any 4 of them. Answer all parts of the questions you select. Place a large x of the page containing the question you are not answering. Do NOT answer all 5. If you do I will only count the 4 worst answers.

This is an individual effort. You may use the class handouts, sources pointed to in the slides, and your notes. Nothing else and no one else. No laptops, no sharing notes or handouts. You have 75 minutes, from 6:30 pm until 7:45 pm.

NOTE***Writing true statements about a topic does not constitute a correct answer to a question.

1. Emergent behavior is the appearance of patterns occurs due to the collective behavior. What emerges cannot be planned or intended. The whole of the interactions becomes greater than the sum of the separate parts. How does a verification process handle these emergent behaviors?
2. How do the assume and guarantee pieces of a design contract show up in test cases?
3. Both black box and white box test cases have the form of <input, execute, expected output>. What makes them different?

a. Integration tests are partly based on flows between components and emergent behavior is a flow on some flow path

b. The assumes contribute test case input specifications and the guarantees provide the output expectations

c. The difference is how test coverage is computed

1. AADL provides a rigorous semantics for modeling. Explain what the complete definition of an end-to-end flow provides that can be used in verifying the correctness of an AADL model.
2. Reqspec allows requirements to be linked to specific architecture structures. What are the advantages to verification from this linkage?
3. What is the difference between system tests and acceptance tests?

a. It provides a complete representation of some requirement

b. The advantage for verification is easier construction of test scenarios and computation of completeness

c. Who executes the tests

3.

a. Using the WBS give a specific example of fault, error, and failure.

b. Describe at least two uses of the error model for a component (designed using EMV2) in product development and test planning.

c. Orthogonal defect classification classifies defects into non-overlapping sets of defect “types.” Explain how ODC is used in a system testing process.

a. Fault – incorrect wheel speed indicator; Error – incorrect braking pressure computation; Failure – runway overrun

b. Test planning must ensure the error model state machine is properly covered; product review must evaluate the transition from nominal to error behavior

c. Ensure that every set of defect types is represented in the test suite if the error type is appropriate for the domain

4.

a. ATAM produces a set of risk themes. Give an example of a risk found during an ATAM. What, if any, changes should the development team make based on this risk being identified?

2. Describe a measure of verification effectiveness. How is it computed?
3. How is the information about the “stimulus source” in a quality attribute used during test planning?

a. Risk: there is a chance the chosen architectural style will not meet the latency requirement. Insert trade studies that examine multiple styles.

b. Number of defects found prior to release/total number of defects found

c. The stimulus source can indicate the priority of the scenario

a. Describe at least two uses of the error model for a component (designed using EMV2) in product development and test planning.

2. Describe a measure of testing process efficiency. How is it computed?
3. How can AADL and OSATE be used during an ATAM?

a. My mistake – duplicate question - Test planning must ensure the error model state machine is properly covered; product review must evaluate the transition from nominal to error behavior

b. Number of defects found/number of test cases

c. The AADL model of flows can be analyzed using OSATE analyses to help with the ATAM search for risks