Skip to main content

Short note: Likelihood approach to anomalies

Short answer

Likelihood anomaly detection models data as a mixture of a normal distribution M and an anomaly distribution A, with D=(1−λ)M+λA. Each object is moved to the anomaly set if doing so increases the data's log-likelihood beyond a threshold. It gives a probabilistic score but requires correct distributions and λ, and struggles in high dimensions.

Write a short note on likelihood approach for anomaly detection.

The likelihood approach (a statistical, probabilistic approach) assumes the data are generated by a mixture of two distributions: a distribution M for the majority (normal) objects and a distribution A for anomalies. Each object is assigned to the distribution under which it is more likely.

Model

The distribution of all data is D = (1 − λ)·M + λ·A, where λ is the small prior probability that an object is an anomaly (e.g. 0.01 to 0.05). The initial assumption is that all objects belong to M (A is empty).

The likelihood of the whole data set is

L(D) = Π over normal objects (1 − λ)·P_M(xᵢ) × Π over anomalies λ·P_A(xᵢ)

and log L is used in practice.

Algorithm

  1. Put all objects into the normal set M_t and A_t = empty. Compute log L_t(D).
  2. For each object x in M_t, temporarily move it to A_t, and recompute the log-likelihood.
  3. If the gain in log-likelihood (the difference) exceeds a threshold c, then x is declared an anomaly and stays in A_t.
  4. Repeat for all objects. The distributions M (e.g. Gaussian estimated from M_t) and A (often uniform) are re-estimated.

An object is an anomaly if moving it out of the normal set increases the likelihood of the data enough, meaning that it does not fit the model of normal data well.

Features

It gives a principled, probabilistic score and a confidence, and can use any distribution family. But it needs a correct choice of distribution for M and A, and λ; it is difficult for high-dimensional data.

✓ Answer checked by Claude Opus 5.5 (high effort)

Related questions

All Data Mining chapter 6 questions →