---
title: "C4.5 vs ID3; gain, gain ratio, Gini"
subject: "Data Mining"
chapter: 3
key: "semester_7/data_mining:c45-gain-ratio-gini"
source: https://www.bctengineer.com/q/semester_7/data_mining/c45-gain-ratio-gini
site: BCT Engineering
---

# C4.5 vs ID3; gain, gain ratio, Gini

Data Mining · Chapter 3: Classification

- Asked in: year not given
- Video: [How to build Decision Tree using C4.5 Algorithm | C4.5 Decision Tree Learning | C4.5 Mahesh Huddar](https://www.youtube.com/watch?v=cmXLhqv67ns) — Mahesh Huddar
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_7/data_mining/c45-gain-ratio-gini
- Whole chapter: https://www.bctengineer.com/md/semester_7/data_mining/chapter-3

## Question

How does C4.5 overcome the problem of ID3 Algorithm? Compare Information Gain, Gain ratio and Gini Index.

## Short answer

C4.5 overcomes ID3's bias toward many-valued attributes by using gain ratio, handles continuous and missing values, and prunes trees. Information gain measures entropy reduction but favors many splits; gain ratio normalizes it; Gini index is a faster impurity measure used in CART.

## Answer

#### How C4.5 overcomes the problems of ID3
C4.5 (Quinlan) is the successor of ID3. It improves ID3 in these ways:

| ID3 problem | C4.5 solution |
| --- | --- |
| Biased to attributes with many values (e.g. an ID attribute) | Uses **gain ratio** = Gain / SplitInfo |
| Only categorical attributes | Handles **continuous** attributes by choosing a threshold (A ≤ t, A > t) |
| Cannot handle missing values | Distributes the record over branches with fractional weights, or ignores it while computing the gain |
| Overfitting, no pruning | **Pruning** (pessimistic error-based post-pruning) |
| Rules not produced | Converts the tree into simplified rules |
| No attribute costs | Allows attributes with different costs |

#### Comparison of splitting measures
- **Information gain** = Entropy(S) − Σ (|Sᵥ|/|S|) Entropy(Sᵥ), where Entropy = −Σ pᵢ log₂ pᵢ.
- **Gain ratio** = Gain(A)/SplitInfo(A), SplitInfo(A) = −Σ (|Sᵥ|/|S|) log₂(|Sᵥ|/|S|).
- **Gini index** = 1 − Σ pᵢ²; for a split, Gini_split = Σ (|Sᵥ|/|S|) Gini(Sᵥ).

| Basis | Information gain | Gain ratio | Gini index |
| --- | --- | --- | --- |
| Used in | ID3 | C4.5 | CART, SLIQ |
| Idea | Reduction in entropy | Gain normalised by split info | Reduction in impurity |
| Choose | Maximum | Maximum | Minimum Gini_split |
| Many-valued bias | Yes | Corrected | Yes (favours multiway), CART uses binary splits |
| Cost | Logarithms | Logarithms | Only squares, faster |

*Example (Play Golf, 9 yes, 5 no, split on Outlook)*: Entropy = 0.940, Gain = 0.247, SplitInfo = 1.577, Gain ratio = 0.156. Gini(S) = 0.459, Gini_split = (5/14)(0.48) + (4/14)(0) + (5/14)(0.48) = 0.343, so Gini reduction = 0.116.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. How does C4.5 fix ID3's bias toward many-valued attributes?
   A. Uses Gini index
   B. Removes the attribute
   C. Uses gain ratio = Gain / SplitInfo
   D. Uses fewer records
   Answer: C. SplitInfo penalises many-valued splits. (Key point: Bias fix)

2. How does C4.5 handle continuous attributes?
   A. Cannot handle them
   B. Discards them
   C. Makes one branch per value
   D. Chooses a threshold (A <= t, A > t)
   Answer: D. ID3 needs discretisation. (Key point: Continuous)

3. In CART, the best split is the one with:
   A. Minimum Gini_split
   B. Maximum Gini_split
   C. Maximum gain ratio
   D. Maximum entropy
   Answer: A. Lower impurity is better. (Key point: Gini choice)

4. In the Outlook example, gain ratio = 0.247/1.577 =
   A. 0.247
   B. 0.156
   C. 1.577
   D. 0.116
   Answer: B. Gain / SplitInfo. (Key point: Gain ratio value)

## Related in Data Mining

- [Entropy, information gain and ID3 tree](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aid3-entropy-information-gain) — Data Mining, chapter 3
- [ID3 algorithm and WEKA confusion matrix](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aid3-algorithm-weka-confusion) — Data Mining, chapter 3
- [ID3 algorithm and why greedy](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aid3-greedy) — Data Mining, chapter 3
- [Overfitting in ID3 and decision tree rules](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aid3-overfitting-rules) — Data Mining, chapter 3
- [ID3 and TP rate, FP rate, precision](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aid3-weka-tp-fp-precision) — Data Mining, chapter 3
- [ID3 steps, overfitting, TP/FP/recall](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aid3-steps-overfitting-recall) — Data Mining, chapter 3

## Related in other subjects

- [Compare industrial age and information age](https://www.bctengineer.com/question.md?key=semester_7%2Ftechnology_environment_and_society%3Acompare-industrial-info-age) — Technology, Environment and Society, chapter 1, asked 2×
- [Modified B/C ratio for the required-receipts problem](https://www.bctengineer.com/question.md?key=semester_6%2Fengineering_economics%3Amodified-bcr-for-5a) — Engineering Economics, chapter 3
- [File system layers; linked list vs index](https://www.bctengineer.com/question.md?key=semester_6%2Foperating_system%3Afs-layers-linked-list-vs-index) — Operating System, chapter 5
- [Detection gain of SSB-SC vs DSB-SC](https://www.bctengineer.com/question.md?key=semester_6%2Fcommunication_systems%3Adetection-gain-ssbsc-dsbsc) — Communication Systems, chapter 9, asked 2×