- Year not given · 3 marks
- 2063 · 3 marks
Short note: Data preprocessing
Short answer
Data preprocessing converts raw data into clean form for mining, addressing incompleteness, noise, inconsistency. Major tasks: cleaning (missing values, noise, outliers), integration (merge sources), transformation (normalization, aggregation), reduction (dimensionality, compression), discretization (numeric to intervals). Takes 60-80% of project effort.
Write a short note on Data Preprocessing.
Data preprocessing is the set of techniques used to convert raw data into a clean and suitable form before data mining. Real data is incomplete (missing values), noisy (errors, outliers) and inconsistent (different codes, units), and low-quality data gives low-quality mining results.
Major tasks
- Data cleaning: fill missing values (ignore tuple, mean, most probable value), smooth noise (binning, regression, clustering), remove outliers, fix inconsistencies.
- Data integration: combine multiple sources; handle schema and entity matching, redundancy (correlation) and value conflicts.
- Data transformation: normalisation (min-max, z-score, decimal scaling), aggregation, generalisation, attribute construction.
- Data reduction: attribute subset selection, PCA, sampling, compression, so data is smaller but gives similar results.
- Data discretisation: convert numeric attributes into intervals or concepts using binning or histograms.
Preprocessing often takes 60 to 80% of the total effort in a data mining project.
✓ Answer checked by Claude Opus 5.5 (high effort)
Related questions
- Data preprocessing and handling noisy data · Data Mining, chapter 2
- Processes in data preprocessing · Data Mining, chapter 2
- Preprocessing steps, binning, normalization · Data Mining, chapter 2