---
title: "Data cleaning: missing, noisy, outliers"
subject: "Data Mining"
chapter: 2
key: "semester_7/data_mining:data-cleaning"
source: https://www.bctengineer.com/q/semester_7/data_mining/data-cleaning
site: BCT Engineering
---

# Data cleaning: missing, noisy, outliers

Data Mining · Chapter 2: Data Pre-processing

- Asked in: year not given
- Video: [Data Cleaning | Missing Values, Noise & Outliers | Data Preprocessing | Lec. 04](https://www.youtube.com/watch?v=0RQJ4dpmUV0) — GateXAIML
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_7/data_mining/data-cleaning
- Whole chapter: https://www.bctengineer.com/md/semester_7/data_mining/chapter-2

## Question

What is data cleaning? Explain different approaches to handle missing data, noisy data and outliers.

## Short answer

Data cleaning corrects errors and inconsistencies. Missing values are handled by deletion, filling with mean/mode, or prediction. Noisy data is smoothed by binning, regression, or clustering. Outliers are detected via statistics or clustering and removed, capped, or kept.

## Answer

**Data cleaning** (data cleansing) is the process of detecting and correcting errors and inconsistencies in data to improve its quality. Real data is incomplete (missing values), noisy (errors and outliers) and inconsistent.

#### 1. Handling missing data
- **Ignore/delete the record**: used when the class label is missing or many attributes are missing; poor when percentage of missing values varies a lot.
- **Fill manually**: accurate but slow and infeasible for large data.
- **Fill with a global constant** such as "Unknown" (may mislead the miner).
- **Fill with the attribute mean or median** (median for skewed data), or the mean of the same class.
- **Fill with the most probable value** predicted by regression, a decision tree or Bayesian inference. This is the best method.

#### 2. Handling noisy data
- **Binning**: sort the values, divide into bins, then smooth by bin means, bin medians or bin boundaries. *Example*: bin (4, 8, 15) becomes (9, 9, 9) by means.
- **Regression**: fit a function to the data and replace values by the fitted values.
- **Clustering**: values falling outside the clusters are noise or outliers.
- **Combined computer and human inspection**: the computer flags suspicious values, a person checks them.

#### 3. Handling outliers
- Detect using statistics (z-score beyond ±3, IQR rule: below Q1 − 1.5·IQR or above Q3 + 1.5·IQR), clustering or distance-based methods, or graphs (box plot).
- Then remove them if they are errors, cap them (winsorising), transform them (log), or keep them if they are genuine and important (as in fraud detection).

Cleaning is iterative: discrepancy detection and data transformation steps are repeated until the data is acceptable.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. Which is the best way to fill missing values?
   A. Always delete the record
   B. Global constant 'Unknown'
   C. Manually filling all values
   D. Most probable value predicted by regression, decision tree or Bayesian inference
   Answer: D. Prediction uses the other attributes and is usually most accurate. (Key point: Best missing value method)

2. For skewed data which is better for filling missing values?
   A. Median
   B. Mean
   C. Mode
   D. Zero
   Answer: A. Skew pulls the mean toward the tail, so median is more representative. (Key point: Mean vs median)

3. Which methods smooth noisy data?
   A. Discretization, sampling, compression
   B. Binning, regression, clustering
   C. Min-max, z-score, decimal scaling
   D. Roll-up, slice, dice
   Answer: B. These smooth or flag noise; computer+human inspection also helps. (Key point: Noisy data methods)

4. Using the IQR rule, a value is an outlier if it is?
   A. Below Q1 or above Q3
   B. Beyond Q1 - IQR to Q3 + IQR
   C. Below Q1 - 1.5*IQR or above Q3 + 1.5*IQR
   D. Equal to the median
   Answer: C. That is the standard box-plot rule. (Key point: IQR outlier rule)

## Related in Data Mining

- [Data preprocessing and handling noisy data](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Apreprocessing-noisy-data) — Data Mining, chapter 2
- [Major clustering approaches](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aclustering-approaches) — Data Mining, chapter 5
- [Graphical drawbacks; statistical approaches](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aanomaly-graphical-statistical) — Data Mining, chapter 6

## Related in other subjects

- [Closed traverse ABCDA, missing data](https://www.bctengineer.com/question.md?key=semester_4%2Fsurveying_ii%3Aclosed-traverse-abcda-missing-2069b) — Surveying II, chapter 1
- [Closed traverse ABCDA missing data, C(1200, 2000)](https://www.bctengineer.com/question.md?key=semester_4%2Fsurveying_ii%3Aclosed-traverse-abcda-c-1200-2000) — Surveying II, chapter 1
- [Level book with missing data, RL 1249.260](https://www.bctengineer.com/question.md?key=semester_3%2Fsurveying_i%3Alevel-book-missing-data-ba-1249-260) — Surveying I, chapter 5
- [Anti-clockwise traverse PQRSP missing data](https://www.bctengineer.com/question.md?key=semester_4%2Fsurveying_ii%3Aanticlockwise-traverse-pqrsp-missing) — Surveying II, chapter 1