Objectives of FTR; cleanroom software engineering

Objectives of Formal Technical Reviews (FTR)

A formal technical review is a software quality control activity in which a small team of engineers (3–5 people) examines a work product in a planned meeting of less than 2 hours, after each reviewer has prepared for about 1–2 hours. Walkthroughs and inspections are types of FTR.

Main objectives (Pressman):

1. Uncover errors in function, logic or implementation in any representation of the software.
2. Verify that the software under review meets its requirements.
3. Ensure that the software is represented according to predefined standards.
4. Achieve software that is developed in a uniform manner across the team.
5. Make projects more manageable by giving checkpoints with clear results.

Additional benefits:

• Defect removal is cheaper early: design errors make up a large share of all errors, and reviews can find most of them before testing, so they do not get amplified into later phases.
• Training ground: junior engineers learn different approaches to analysis, design and coding.
• Backup and continuity: more people become familiar with parts of the software they did not write.

Outcome of an FTR: the team decides to accept, reject, or accept provisionally, and produces a review issues list and a review summary report. The review leader follows up to make sure the issues are corrected.

Cleanroom software engineering

Cleanroom software engineering is a formal approach to software development whose aim is to prevent defects rather than find and remove them later. The name comes from cleanrooms used in semiconductor manufacturing, where dust is kept out instead of being cleaned up afterwards. It was developed by Harlan Mills and colleagues at IBM.

Key strategies (Sommerville):

1. Formal specification: the software is specified using a formal model, such as a state-transition model of how the system responds to stimuli.
2. Incremental development: the system is divided into increments that are developed and certified separately, with critical functions in early increments.
3. Structured programming: only a limited set of control and data abstraction constructs is used; the program is developed by stepwise refinement of the specification.
4. Static verification: the code is checked by rigorous inspections and mathematical correctness arguments. Developers do not unit test or execute their code.
5. Statistical testing: each integrated increment is tested statistically using an operational profile to estimate its reliability.

Pressman describes the same ideas using box structure specification: a black box (external behaviour), a state box (state and data) and a clear box (procedural design), followed by correctness verification of each box.

Teams

• Specification team: develops and maintains the formal system specification.
• Development team: develops and verifies the software (no execution).
• Certification team: develops statistical tests from the specification and certifies the reliability.

 Formally specify --> Define increments
        |                   |
        v                   v
 Develop operational    Construct structured
 profile                program
        |                   |
        v                   v
 Design statistical     Formally verify code
 tests                      |
        |                   v
        +-------> Integrate increment
                            |
                            v
                Test integrated system
                (certify reliability)

Advantages

• Very low defect density in delivered software; IBM-reported projects had only a few errors per thousand lines of code.
• Reliability is certified with statistical evidence.
• Lower testing and maintenance cost.

Limitations

• Needs skilled staff trained in formal methods.
• Formal specification and verification take time and effort at the start.
• Less suitable for projects with unstable or unclear requirements, such as user-interface-heavy systems.

Difference from conventional development: conventional practice relies on unit testing and debugging to remove defects, while cleanroom relies on correct design and verification so that defects are not introduced in the first place.