---
title: "Avoiding curse of dimensionality"
subject: "Data Mining"
chapter: 2
key: "semester_7/data_mining:avoid-curse-dimensionality"
source: https://www.bctengineer.com/q/semester_7/data_mining/avoid-curse-dimensionality
site: BCT Engineering
---

# Avoiding curse of dimensionality

Data Mining · Chapter 2: Data Pre-processing

- Asked in: year not given
- Video: [Curse Of Dimensionality Explained with Examples in Hindi ll Machine Learning Course](https://www.youtube.com/watch?v=SMaa3pnQmbg) — 5 Minutes Engineering
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_7/data_mining/avoid-curse-dimensionality
- Whole chapter: https://www.bctengineer.com/md/semester_7/data_mining/chapter-2

## Question

Explain why and how to avoid curse of dimensionality.

## Short answer

The curse must be avoided because high-dimensional data degrades mining performance. Avoid it by feature selection (keeping relevant attributes), feature extraction (e.g., PCA), increasing sample size, using robust algorithms, and applying domain knowledge to reduce dimensions.

## Answer

#### Why it must be avoided
In high dimensions data become sparse, distances between objects look nearly equal, and classifiers, clusterers and outlier detectors perform poorly. Models overfit, need many more training records, and take more time and memory. Avoiding the curse gives better accuracy, faster mining and results that people can understand and visualise.

#### How to avoid it
1. **Feature selection**: keep only relevant attributes and drop redundant (highly correlated) and irrelevant ones.
   - *Filter* methods (correlation, information gain, chi-square)
   - *Wrapper* methods (try subsets with a classifier: forward selection, backward elimination)
   - *Embedded* methods (e.g. decision-tree importance, L1 regularisation)
2. **Feature extraction / projection**: create a few new attributes from the original ones.
   - Principal Component Analysis (PCA)
   - Singular Value Decomposition (SVD), Linear Discriminant Analysis (LDA), wavelet transform
3. **Get more data**: increase the number of training samples relative to the number of features.
4. **Use suitable algorithms/measures**: methods that tolerate high dimensions (e.g. regularised models, cosine similarity for sparse text), or subspace clustering.
5. **Domain knowledge**: choose attributes meaningful for the problem.
6. **Aggregation/discretization** to merge attributes or values.

Rule of thumb: the number of records should be much larger than the number of attributes, and attribute selection should be done before the mining algorithm is applied.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. What does feature selection do?
   A. Keeps relevant attributes and drops redundant and irrelevant ones
   B. Creates new attributes from combinations
   C. Adds attributes
   D. Increases samples only
   Answer: A. It keeps a subset of original attributes. (Key point: Feature selection)

2. Which is a filter method?
   A. Forward selection using a classifier
   B. Information gain or chi-square scoring of each attribute
   C. L1 regularisation during training
   D. PCA projection
   Answer: B. Filters score attributes independently; forward selection is a wrapper and L1 is embedded. (Key point: Filter methods)

3. Which technique projects the data onto fewer new attributes?
   A. Chi-square filter
   B. Backward elimination
   C. PCA
   D. Stratified sampling
   Answer: C. PCA creates a few new attributes from the original ones. (Key point: Feature extraction)

4. What rule of thumb is given regarding records and attributes?
   A. Attributes should exceed records
   B. They should be equal
   C. Records do not matter
   D. Number of records should be much larger than the number of attributes
   Answer: D. More records relative to features reduce overfitting and sparsity. (Key point: Rule of thumb)

## Related in Data Mining

- [Dimension of data; curse of dimensionality](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Adimension-curse) — Data Mining, chapter 2

## Related in other subjects

- [Avoiding faults; checkpointing compared](https://www.bctengineer.com/question.md?key=semester_7%2Fdistributed_system%3Aavoid-faults-checkpointing) — Distributed System, chapter 9
- [Routing factors and avoiding crosstalk](https://www.bctengineer.com/question.md?key=semester_5%2Finstrumentation_ii%3Aroute-factors-crosstalk) — Instrumentation II, chapter 8
- [Avoiding load shedding and approach to 2020](https://www.bctengineer.com/question.md?key=semester_7%2Fhydropower_engineering%3Aload-shedding-approach-2020) — Hydropower Engineering, chapter 1