Software reliability and reliability models

Software reliability

Software reliability is the probability of failure-free operation of a computer program in a specified environment for a specified time (Musa). For example, if a program has a reliability of 0.97 over 8 hours of operation, then in 100 runs of 8 hours each it is expected to work without failure in about 97 of them.

Common reliability metrics:

• MTTF (mean time to failure) and MTBF = MTTF + MTTR (mean time to repair).
• Availability = MTTF / (MTTF + MTTR) × 100%.
• ROCOF (rate of occurrence of failures) and POFOD (probability of failure on demand).

If the failure rate λ is constant, reliability over time t is R(t) = e^(−λt). For λ = 0.002 failures/hour and t = 100 hours, R = e^(−0.2) ≈ 0.82.

Software reliability models

Software does not wear out; its reliability improves as faults are found and removed during testing and use. Reliability growth models describe this improvement and are used to predict when the required reliability will be reached, so the team knows when to stop testing.

1. Equal-step function model (Jelinski–Moranda)

• Assumes the program starts with N faults, every fault contributes equally to the failure rate, and each fix removes exactly one fault without adding new ones.
• So reliability increases by an equal step after every repair.
• Failure rate after (i − 1) faults are removed: λi = φ (N − i + 1), where φ is the failure rate contributed by one fault.

Example: N = 50 faults, φ = 0.01 failures/hour per fault.

• At the start: λ = 0.01 × 50 = 0.5 failures/hour, MTTF = 1/0.5 = 2 hours.
• After 30 faults are removed: λ = 0.01 × 20 = 0.2 failures/hour, MTTF = 5 hours.

2. Random-step function model (Littlewood–Verrall)

• More realistic: fixing a fault changes reliability by a random amount, because faults differ in how often they occur, and a repair may introduce new faults.
• Reliability usually rises, but sometimes it falls after a fix.

Reliability              Equal-step model:
  ^              ____    each fix raises
  |         ____|        reliability by the
  |    ____|             same amount
  |___|
  +------------------> faults fixed

Reliability              Random-step model:
  ^             __       each fix changes
  |      __    |  |__    reliability by a
  |  ___|  |___|         random amount; it
  |_|                    may even fall
  +------------------> faults fixed

3. Continuous (exponential) models: Musa's basic execution time model

• Failure intensity falls linearly with the number of failures experienced: λ(μ) = λ0 (1 − μ / ν0) where λ0 = initial failure intensity, μ = failures experienced so far, ν0 = total failures expected in infinite time.

Example: λ0 = 10 failures/CPU hour, ν0 = 100. After μ = 50 failures, λ = 10 × (1 − 50/100) = 5 failures/CPU hour.

4. Logarithmic Poisson model (Musa–Okumoto)

• Early fixes give a large improvement and later fixes give smaller ones: λ(μ) = λ0 e^(−θμ), where θ is the failure intensity decay parameter.

Example: λ0 = 10, θ = 0.02, μ = 50: λ = 10 × e^(−1) ≈ 3.68 failures/CPU hour.

Other classification

• Prediction models use historical data from earlier projects, before testing starts.
• Estimation models use failure data from the current project's testing.

Use in practice: the system is tested with an operational profile (inputs in the same proportion as real use), failure times are recorded, a model is fitted to the data, and the predicted reliability is compared with the required value (for example ROCOF ≤ 2 failures per 1000 hours) to decide whether to release.