---
title: "Short note: Likelihood approach to anomalies"
subject: "Data Mining"
chapter: 6
key: "semester_7/data_mining:likelihood-anomaly-note"
source: https://www.bctengineer.com/q/semester_7/data_mining/likelihood-anomaly-note
site: BCT Engineering
---

# Short note: Likelihood approach to anomalies

Data Mining · Chapter 6: Anomaly / Fraud Detection

- Asked in: year not given
- Video: [Anomaly Detection in Machine Learning | Gaussian Distribution Explained (Step-by-Step)](https://www.youtube.com/watch?v=HfzN9wGhU_A) — Coursesteach
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_7/data_mining/likelihood-anomaly-note
- Whole chapter: https://www.bctengineer.com/md/semester_7/data_mining/chapter-6

## Question

Write a short note on likelihood approach for anomaly detection.

## Short answer

Likelihood anomaly detection models data as a mixture of a normal distribution M and an anomaly distribution A, with D=(1−λ)M+λA. Each object is moved to the anomaly set if doing so increases the data's log-likelihood beyond a threshold. It gives a probabilistic score but requires correct distributions and λ, and struggles in high dimensions.

## Answer

The **likelihood approach** (a statistical, probabilistic approach) assumes the data are generated by a mixture of two distributions: a distribution **M** for the majority (normal) objects and a distribution **A** for anomalies. Each object is assigned to the distribution under which it is more likely.

#### Model
The distribution of all data is D = (1 − λ)·M + λ·A, where λ is the small prior probability that an object is an anomaly (e.g. 0.01 to 0.05). The initial assumption is that all objects belong to M (A is empty).

The likelihood of the whole data set is

L(D) = Π over normal objects (1 − λ)·P_M(xᵢ) × Π over anomalies λ·P_A(xᵢ)

and log L is used in practice.

#### Algorithm
1. Put all objects into the normal set M_t and A_t = empty. Compute log L_t(D).
2. For each object x in M_t, temporarily move it to A_t, and recompute the log-likelihood.
3. If the **gain** in log-likelihood (the difference) exceeds a threshold c, then x is declared an **anomaly** and stays in A_t.
4. Repeat for all objects. The distributions M (e.g. Gaussian estimated from M_t) and A (often uniform) are re-estimated.

An object is an anomaly if moving it out of the normal set increases the likelihood of the data enough, meaning that it does not fit the model of normal data well.

#### Features
It gives a principled, probabilistic score and a confidence, and can use any distribution family. But it needs a correct choice of distribution for M and A, and λ; it is difficult for high-dimensional data.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. In the likelihood approach, how are data modelled?
   A. D = (1 - lambda)M + lambda A, a mixture of normal M and anomaly A distributions
   B. As a single Gaussian only
   C. As a dendrogram
   D. As grid cells
   Answer: A. Normal objects come from M and anomalies from A, with small prior lambda. (Key point: Mixture model)

2. What does the algorithm initially assume?
   A. All objects are anomalies
   B. All objects belong to M and A is empty
   C. Half are anomalies
   D. Anomalies equal lambda*n
   Answer: B. It starts with all objects in M_t and A_t empty. (Key point: Initial assumption)

3. When is an object declared an anomaly?
   A. When moving it from M to A increases the log-likelihood by more than a threshold c
   B. When it is the largest value
   C. When its likelihood is exactly 1
   D. When lambda equals 0
   Answer: A. A large gain in log-likelihood means the object does not fit the normal model. (Key point: Decision rule)

## Related in Data Mining

- [Anomaly detection and importance](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aanomaly-detection-intro) — Data Mining, chapter 6
- [Anomaly detection challenges and schemes](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aanomaly-challenges-schemes) — Data Mining, chapter 6
- [Short note: Distance-based anomaly detection](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Adistance-based-anomaly-note) — Data Mining, chapter 6

## Related in other subjects

- [Limitations of back-face detection](https://www.bctengineer.com/question.md?key=semester_5%2Fcomputer_graphics%3Aback-face-limitations-overcome) — Computer Graphics, chapter 7, asked 2×