---
title: "Data preprocessing and handling noisy data"
subject: "Data Mining"
chapter: 2
key: "semester_7/data_mining:preprocessing-noisy-data"
source: https://www.bctengineer.com/q/semester_7/data_mining/preprocessing-noisy-data
site: BCT Engineering
---

# Data preprocessing and handling noisy data

Data Mining · Chapter 2: Data Pre-processing

- Asked in: 2066 Bhadra, 4+6 marks; year not given, 4+4 marks; year not given, 4+6 marks; year not given
- Video: [Data Preprocessing And Noisy Data (& It"s Technique ) || Data Warehousing and Data Mining](https://www.youtube.com/watch?v=Fo3X1wbwka4) — Let's Learn With Us
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_7/data_mining/preprocessing-noisy-data
- Whole chapter: https://www.bctengineer.com/md/semester_7/data_mining/chapter-2

## Question

What kind of data preprocessing do we need before applying a data mining algorithm to any dataset? Explain the techniques (including the binning method) to handle noisy data.

## Short answer

Preprocessing tasks: cleaning, integration, transformation, reduction, discretization. Noisy data handled by binning (sort, partition, smooth by bin means/medians/boundaries), regression, clustering, or human inspection. Example: equal-frequency binning of prices smooths values within each bin.

## Answer

#### Data preprocessing
Real-world data is dirty (incomplete, noisy, inconsistent), and poor data gives poor mining results. Preprocessing makes the data fit for mining. The major tasks are:
1. **Data cleaning**: fill missing values, smooth noisy data, identify outliers, resolve inconsistencies.
2. **Data integration**: merge data from several sources, remove redundancy.
3. **Data transformation**: normalisation, aggregation, generalisation, attribute construction.
4. **Data reduction**: reduce volume but keep the analytical result: dimensionality reduction, numerosity reduction, compression.
5. **Data discretisation**: replace numeric values by intervals or concept labels.

#### Noisy data and the techniques to handle it
**Noise** is random error or variance in a measured variable (e.g. age = 250).

1. **Binning**: sort the data and divide it into bins (equal-frequency or equal-width); then smooth each value by its bin neighbours (local smoothing).
2. **Regression**: fit the data to a function (linear/multiple regression); values are smoothed to the fitted line.
3. **Clustering**: group similar values; values outside the clusters are outliers and are removed.
4. **Combined computer and human inspection**: the computer flags suspicious values, a human checks them.

#### Binning example
Price data (sorted): 4, 8, 15, 21, 21, 24, 25, 28, 34. Use equal-frequency bins of depth 3.

| Bin | Values | Smoothing by means | By median | Smoothing by boundaries |
|---|---|---|---|---|
| 1 | 4, 8, 15 | 9, 9, 9 | 8, 8, 8 | 4, 4, 15 |
| 2 | 21, 21, 24 | 22, 22, 22 | 21, 21, 21 | 21, 21, 24 |
| 3 | 25, 28, 34 | 29, 29, 29 | 28, 28, 28 | 25, 25, 34 |

Means: (4+8+15)/3 = 9; (21+21+24)/3 = 22; (25+28+34)/3 = 29. In smoothing by boundaries, each value moves to the closer of the bin's minimum and maximum (8 is 4 from 4 and 7 from 15, so 8 → 4).

Larger bin width gives stronger smoothing.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. Bin 4, 8, 15 smoothed by means becomes:
   A. 8, 8, 8
   B. 9, 9, 9
   C. 4, 4, 15
   D. 15, 15, 15
   Answer: B. (4+8+15)/3 = 9. (Key point: Smoothing by means)

2. In smoothing by boundaries, 8 in bin (4, 8, 15) becomes:
   A. 15
   B. 4
   C. 8
   D. 9
   Answer: B. 8 is 4 from 4 and 7 from 15, so it moves to 4. (Key point: Smoothing by boundaries)

3. Noise is:
   A. Missing data
   B. Random error or variance in a measured variable
   C. Duplicate schemas
   D. A hierarchy
   Answer: B. Example: age = 250. (Key point: Noise definition)

4. Clustering handles noise by:
   A. Filling missing values
   B. Detecting values outside clusters as outliers and removing them
   C. Sorting only
   D. Normalising
   Answer: B. Values falling outside clusters are outliers. (Key point: Clustering for noise)

## Related in Data Mining

- [Preprocessing steps, binning, normalization](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Apreprocessing-binning-normalization) — Data Mining, chapter 2
- [Data cleaning: missing, noisy, outliers](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Adata-cleaning) — Data Mining, chapter 2
- [Short note: Data preprocessing](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Adata-preprocessing-note) — Data Mining, chapter 2
- [Processes in data preprocessing](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Apreprocessing-processes) — Data Mining, chapter 2
- [Association mining on varied data types](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aassoc-categorical-sequential-stream) — Data Mining, chapter 4
- [Web mining, importance and techniques](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aweb-mining-importance-techniques) — Data Mining, chapter 7

## Related in other subjects

- [Handling voluminous data](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Avoluminous-data-techniques-challenges) — Information System, chapter 8
- [Why do we need DFT](https://www.bctengineer.com/question.md?key=semester_7%2Fdigital_signal_analysis_and_processing%3Awhy-need-dft) — Digital Signal Analysis and Processing, chapter 7, asked 6×
- [Why do we need FFT](https://www.bctengineer.com/question.md?key=semester_7%2Fdigital_signal_analysis_and_processing%3Awhy-need-fft) — Digital Signal Analysis and Processing, chapter 7, asked 2×
- [Data mining and business knowledge](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Adata-mining-business-knowledge) — Information System, chapter 4