Skip to main content

Calibration and validation of models

Short answer

Calibration tunes a model by comparing output with reality and adjusting until it matches. Validation ensures the model accurately represents the real system. For a petrol station, calibration changed service time to normal and adjusted arrival rate until average wait matched observed 4.6 min; validation on separate data confirmed accuracy.

What is calibration and validation of models? Explain with practical example.

Calibration

Calibration is the iterative process of comparing the simulation model with the real system and adjusting the model (parameters, distributions, logic) until the differences are acceptably small. Each cycle is: compare, find the discrepancy, revise, compare again.

Validation

Validation is the process of determining whether the model is an accurate representation of the real system for the intended purpose. Its question is "Are we building the right model?" (verification, in contrast, asks "Are we building the model right?", i.e. is the program free of coding errors). Validation is done by:

  1. Face validity: experts in the system review the model and its output for reasonableness.
  2. Validating model assumptions: structural assumptions (how the system works) and data assumptions (input distributions are fitted and tested, e.g. by the chi-square or K-S test).
  3. Validating input-output transformations: the model output is compared statistically with the real output under the same input conditions (e.g. a t-test on the mean, or confidence intervals).

Practical example: single-pump petrol station

A model is built with Poisson arrivals (mean 10 vehicles per hour) and exponential service time (mean 4 min). The data collected from the real station show an average waiting time of 4.6 minutes.

  • Calibration: the model gives about 8 minutes (as M/M/1 theory also predicts for ρ = 10 × 4/60 = 0.67). Checking the data shows that service time is not exponential; it is better fitted by a normal distribution with mean 4 min and standard deviation 1 min. Less variable service shortens the queue, and after the change the model gives about 4.3 minutes. Adjusting the arrival rate for peak hours brings it to 4.5 minutes.
  • Validation: the calibrated model is run with a new set of observations (another week's data) that was not used for calibration. The simulated mean waiting time of 4.5 min, with a 95% confidence interval of 4.0–5.0 min, contains the real value of 4.6 minutes. The two agree statistically, and the managers confirm that the queue lengths look realistic (face validity). So the model is considered valid for studying whether a second pump is needed.

Both steps together increase the credibility of the model, so that decision makers will use its results.

✓ Answer checked by Claude Opus 5.5 (high effort)

Related questions

All Simulation and Modeling chapter 7 questions →