Estimation methods in output analysis

Because a simulation output is a random sample, a performance measure θ (such as mean waiting time) can only be estimated. Estimation methods turn the output data into a point estimate and an interval estimate with a stated confidence.

Point estimate

For n observations Y_1 ... Y_n, the sample mean Ȳ = ΣY_i / n estimates θ. Sample variance S² = Σ(Y_i - Ȳ)² / (n - 1) measures variability. For a time-persistent measure, use the time-weighted average.

Interval estimate (confidence interval)

For terminating simulations, make R independent replications, giving one summary value Y_r from each. Then

Ȳ ± t(α/2, R-1) · S / √R

where t is from the Student's t-table. The interval says that with probability 1 - α the true mean lies inside it.

Estimation for steady state

Within one long run the observations are correlated, so the usual formula is invalid. Methods used:

• Replication with deletion: independent replications with warm-up removed.
• Batch means: split the run into k batches; use batch means as nearly independent observations in the formula above.
• Regenerative method: use cycles that start from the same state (such as an empty system).

Example

Five replications of a bank model give average delays: 2.1, 2.5, 1.9, 2.8, 2.2 min.

• Ȳ = 11.5 / 5 = 2.30
• S² = (0.04 + 0.04 + 0.16 + 0.25 + 0.01) / 4 = 0.125, S = 0.354
• For 95% confidence, α = 0.05, degrees of freedom = 4, t(0.025, 4) = 2.776
• Half-width = 2.776 × 0.354 / √5 = 0.44
• 95% CI: 2.30 ± 0.44, i.e. (1.86, 2.74) min

Precision

To get half-width ε, the required replications are about R ≥ (t · S / ε)². For ε = 0.1, R is about 50. A narrow interval means a precise estimate; the interval is narrowed by more replications or longer runs.