Model building, verification, validation steps

Model building, verification and validation form one connected process. A model is first built from the real system, the program is verified against the conceptual model, and the final model is validated against the real system, with calibration repeated until the model is credible.

Steps in model building

1. Observe the real system and study how it works, including its interactions among components.
2. Collect and analyse data: record arrival and service times, fit distributions, and list assumptions.
3. Build the conceptual model: components, entities, state variables, events, level of detail and the structure of logic.
4. Translate to an operational (computer) model using a language or package.

Verification steps

• Ensure the program matches the conceptual model.
• Have another person read the code (structured walk-through).
• Trace entities and events by hand and compare with the output.
• Run with simple inputs for which the answer is known, such as a deterministic arrival and service time.
• Use a debugger, print state variables, and check that input parameters are read and used correctly.
• Check that output is reasonable when inputs are varied.

Validation steps (Naylor and Finger)

1. Get high face validity from users and experts.
2. Validate assumptions (structure and data).
3. Compare the model's input-output relationship with the real system's (statistical test or confidence interval).
4. If the model is not accurate enough, calibrate: change the model and repeat.

Diagram

   Real system
   |         ^
   | abstract|  validation +
   v         |  calibration
 Conceptual  |
   model     |
   |         |
   | verification (program matches
   v          conceptual model)
 Operational model (program)
   |
   +-----> run, compare with real data

Iteration

If the comparison fails, go back to data, assumptions or logic, correct them and repeat. Only a model that has been verified and validated should be used for experiments and decisions.