Final Exam Key

CpSc 873

Verification and Validation

 

Name ____________________________________

 

There are 5 questions. Answer any 4 of them. Answer all parts of the questions you select. Place a large x of the page containing the question you are not answering. Do NOT answer all 5. If you do I will only count the 4 worst answers.

 

This is an individual effort. You may use the class handouts, sources pointed to in the slides, and your notes. Nothing else and no one else. No laptops, no sharing notes or handouts. You have 75 minutes, from 3 pm until 4:15 pm.

 

 

 

 

  1.  
    1. Continuous integration allows a developer to get feedback about whether their work “fits” with the work of other developers. Explain how testing is integrated into a development process that is using continuous integration.
    2. Describe how an issue tracking tool is used by V&V personnel. How is it used by development personnel?
    3. How is a project risk analysis used by testing personnel?

 

a.      Testing is automated; a set of tests are written and grouped into test suites that represent specific needs such as regression testing

b.     Test failures, ambiguous specifications and other notes are entered as communication to the development team; similarly communication is sent back. The issue tracker provides a record of the communication for traceability

c.      Testers will use more thorough coverage levels for items that pose  a higher risk

 

  1.  
    1. To calculate the reliability of a piece of software we need to know its operational profile. Describe how a change in the operational profile for a software product can affect the reliability of that software.
    2. AADL and supporting annexes and languages emphasize traceability. How is traceability supported between the architecture definition in AADL and the requirements described in reqspec?
    3. In linear temporal logic “p until q” states that p is true until q becomes true. How do we use testing to investigate whether this is a true statement?

 

 

a.      The perceived reliability may change depending on whether there are features that work but are now used less or whether those features are used more.

b.     The “For” statement is used to refer to the system element covered by the requirements in that set.

c.      We run a set of tests in which after varying lengths of time we set q true. If p has remained true the test passes.

3.

a.      Testability is the degree to which the software under test can be controlled and observed. Explain the trade-off between testability and modularity.

b.     Defect density is a useful measure of quality but describe how it is useful in the testing process.

c.      What techniques do we have for identifying hazards? What techniques do we have for documenting them?

 

 

 

d.     The more modular the less observable since a module encapsulates

e.      As we test if we are not  finding faults at the rate indicated by the defect density we should reexamine our test selection and test coverage criteria.

f.      Expert opinion and prior incidents are the two best ways but also models like Leveson’s control theory.  A fault hazard analysis builds trees of hazards and their causes  

4.

a.      Explain the meaning of each of the following lines:

annex agree {**

           assume "A input range" : Input < 20;

           guarantee "A output range" : Output < 2*Input;

   **};

 

b.     Give two examples of faults in a real-time system. Explain how they would be in different categories in an orthogonal defect classification.

c.      How does the definition of coverage change from one type of industry to another? From one programming language to another?

 

 

 

 

a.      The annex statement opens a special environment; the assume sets a value for Input that is treated as invariant; the guarantee is the test to be met

b.     Late delivery of data and missing data; they are in different hierarchies in the  EMV annex

c.      The more life critical the domain the more in-depth the coverage must be.

5.

 

a.      We separate specification from implementation because we want to have the option to have multiple implementations per specification. What implications does this have for the structure of test suites?

b.     A tester is given a “unit” to test. Describe the steps the tester should take to test it.  Include any other artifacts that you need before designing the test suite.

c.      What can be done to the software design of a product to enhance testability?

 

 

 

a.      Tests can be reused by keeping specification-based tests separate from implementation-based

b.     (1) analyze the specification of the unit;(2) construct tests for each feature in the spec; (3) run the suite; (4) evaluate coverage; (5) analyze the implementation ;  (6) construct tests that will complete the overall coverage of the unit; (7) run the combined test suite

c.      Remove information hiding among modules; add methods that set state values to control the product