---
title: "Dimensionality reduction techniques"
subject: "Data Mining"
chapter: 2
key: "semester_7/data_mining:dimensionality-reduction"
source: https://www.bctengineer.com/q/semester_7/data_mining/dimensionality-reduction
site: BCT Engineering
---

# Dimensionality reduction techniques

Data Mining · Chapter 2: Data Pre-processing

- Asked in: year not given
- Video: [Dimensionality Reduction Importance and Types in Machine Learning by Mahesh Huddar](https://www.youtube.com/watch?v=99SBxEDVfZo) — Mahesh Huddar
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_7/data_mining/dimensionality-reduction
- Whole chapter: https://www.bctengineer.com/md/semester_7/data_mining/chapter-2

## Question

Explain the techniques for dimensionality reduction.

## Short answer

Dimensionality reduction reduces attributes to simplify data and avoid the curse. Techniques include feature extraction (PCA, SVD, LDA, wavelets) that create new attributes, and feature selection (filter, wrapper, embedded) that keeps a subset of original attributes.

## Answer

**Dimensionality reduction** reduces the number of attributes (dimensions) so that the data become easier to store, visualise and mine, and the curse of dimensionality is avoided. There are two main families.

#### 1. Feature extraction (projection) techniques
New attributes are built as combinations of the old ones.
- **PCA (Principal Component Analysis)**: finds orthogonal directions (principal components) of maximum variance. Steps: normalise the data, compute the covariance matrix, find its eigenvectors/eigenvalues, sort eigenvalues in descending order, keep the top *k* eigenvectors, and project the data on them. Keeping components that explain, say, 95% of the variance gives a much smaller data set.
- **SVD (Singular Value Decomposition)**: factors the data matrix A = UΣVᵀ and keeps the largest singular values.
- **LDA (Linear Discriminant Analysis)**: supervised; finds directions that best separate classes.
- **Wavelet transform** (DWT): keeps only the strongest wavelet coefficients; good for signals and images.
- **Non-linear methods**: t-SNE, Isomap, autoencoders.

#### 2. Feature selection techniques
A subset of the original attributes is kept.
- **Filter**: score each attribute independently (information gain, chi-square, correlation) and pick the best.
- **Wrapper**: evaluate subsets by the accuracy of a mining model (forward selection, backward elimination, decision-tree induction).
- **Embedded**: selection happens during training (L1/Lasso, tree importance).
- Removing attributes with low variance or high mutual correlation.

#### Result
Smaller data, faster algorithms, less noise and overfitting, and better visualisation. The price is some loss of information, so the reduction should keep the structure needed for the task.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. Which are the two main families of dimensionality reduction?
   A. Binning and clustering
   B. Sampling and compression
   C. Feature extraction (projection) and feature selection
   D. Roll-up and drill-down
   Answer: C. Extraction builds new attributes; selection keeps a subset. (Key point: Two families)

2. In PCA, which components are kept?
   A. Eigenvectors with smallest eigenvalues
   B. All eigenvectors
   C. Random columns
   D. The top k eigenvectors with the largest eigenvalues of the covariance matrix
   Answer: D. Components are sorted by eigenvalue (variance) and the top k are used to project the data. (Key point: PCA steps)

3. How does LDA differ from PCA?
   A. LDA is supervised and finds directions that best separate classes
   B. LDA is unsupervised and maximises variance
   C. LDA selects subsets of original features
   D. LDA is a sampling method
   Answer: A. PCA maximises variance without labels; LDA uses class labels. (Key point: LDA)

4. What is a wrapper feature selection method?
   A. Scores each attribute independently
   B. Evaluates subsets by the accuracy of a mining model
   C. Selection during training such as Lasso
   D. Projects data on eigenvectors
   Answer: B. Wrappers such as forward selection test subsets with a model. (Key point: Wrapper method)

## Related in other subjects

- [Risk transfer, reduction as response techniques](https://www.bctengineer.com/question.md?key=semester_7%2Fproject_engineering%3Arisk-transfer-reduction-response-justify) — Project Engineering, chapter 5, asked 2×
- [Block diagram reduction with G1, G2, H1, H2, feedforward](https://www.bctengineer.com/question.md?key=semester_3%2Fcontrol_systems%3Abdr-g1g2-h1h2-ff-unity) — Control Systems, chapter 3, asked 2×