Skip to main content

Avoiding curse of dimensionality

Short answer

The curse must be avoided because high-dimensional data degrades mining performance. Avoid it by feature selection (keeping relevant attributes), feature extraction (e.g., PCA), increasing sample size, using robust algorithms, and applying domain knowledge to reduce dimensions.

Explain why and how to avoid curse of dimensionality.

Why it must be avoided

In high dimensions data become sparse, distances between objects look nearly equal, and classifiers, clusterers and outlier detectors perform poorly. Models overfit, need many more training records, and take more time and memory. Avoiding the curse gives better accuracy, faster mining and results that people can understand and visualise.

How to avoid it

  1. Feature selection: keep only relevant attributes and drop redundant (highly correlated) and irrelevant ones.
    • Filter methods (correlation, information gain, chi-square)
    • Wrapper methods (try subsets with a classifier: forward selection, backward elimination)
    • Embedded methods (e.g. decision-tree importance, L1 regularisation)
  2. Feature extraction / projection: create a few new attributes from the original ones.
    • Principal Component Analysis (PCA)
    • Singular Value Decomposition (SVD), Linear Discriminant Analysis (LDA), wavelet transform
  3. Get more data: increase the number of training samples relative to the number of features.
  4. Use suitable algorithms/measures: methods that tolerate high dimensions (e.g. regularised models, cosine similarity for sparse text), or subspace clustering.
  5. Domain knowledge: choose attributes meaningful for the problem.
  6. Aggregation/discretization to merge attributes or values.

Rule of thumb: the number of records should be much larger than the number of attributes, and attribute selection should be done before the mining algorithm is applied.

✓ Answer checked by Claude Opus 5.5 (high effort)

Related questions

All Data Mining chapter 2 questions →