Need for replications of runs

A replication is a complete, independent run of the simulation using the same model and input parameters but a different stream of random numbers (different seeds). Replications are needed because one run gives only one random sample of the output.

Reasons for replications

1. Randomness: the output of a run is a random variable. One run may be unusually good or bad, and its value alone is not reliable.
2. Independent observations: observations inside one run are autocorrelated (a long queue now means a long queue a moment later), so the classical formulas for variance and confidence intervals do not apply. Averages of whole runs are independent and identically distributed.
3. Confidence intervals: with R replications, Ȳ ± t(α/2, R-1)·S/√R can be computed. A single run gives no estimate of variance.
4. Precision control: more replications reduce the half-width in proportion to 1/√R.
5. Initial bias: each replication starts afresh, so the effect of start-up conditions can be examined, and a warm-up can be applied.
6. Comparing alternatives: reliable comparison of two designs needs several runs of each.

Conditions

• Same model, parameters and run length in every replication.
• Different random-number seeds or non-overlapping streams, so that runs are independent.
• Run length and number of replications chosen from the required accuracy.

Example

Mean customer delay in a single-teller bank model from one run is 2.1 min. Four more runs with different seeds give 2.5, 1.9, 2.8 and 2.2 min. The single value 2.1 hides the variation; the five runs show Ȳ = 2.30 and S = 0.354. The 95% confidence interval is 2.30 ± 2.776 × 0.354/√5 = 2.30 ± 0.44, i.e. (1.86, 2.74) min. A manager can now see that the true delay may be anywhere from about 1.9 to 2.7 min, which is not visible from one run.

For a half-width of 0.1 min, about (1.96 × 0.354 / 0.1)² ≈ 48, so around 50 replications are required.