---
title: "ID3 algorithm and WEKA confusion matrix"
subject: "Data Mining"
chapter: 3
key: "semester_7/data_mining:id3-algorithm-weka-confusion"
source: https://www.bctengineer.com/q/semester_7/data_mining/id3-algorithm-weka-confusion
site: BCT Engineering
---

# ID3 algorithm and WEKA confusion matrix

Data Mining · Chapter 3: Classification

- Asked in: year not given, 6+3 marks
- Video: [Creating Decision Tree Using ID3 and J48 in Weka 3.6.3 Tool for Data Mining](https://www.youtube.com/watch?v=OEiQpWn586Q) — Deep Baldha
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_7/data_mining/id3-algorithm-weka-confusion
- Whole chapter: https://www.bctengineer.com/md/semester_7/data_mining/chapter-3

## Question

Write the ID3 algorithm. Explain the significance of the confusion matrix generated by WEKA.

## Short answer

ID3 builds a tree top-down by selecting the highest information gain attribute, branching for each value, and recursing until all examples have one class, no attributes remain, or a branch is empty. The WEKA confusion matrix compares actual versus predicted classes; the diagonal shows correct counts, off-diagonal cells show errors, and it supports accuracy, TP rate, FP rate, precision, recall and F-measure.

## Answer

#### ID3 algorithm
ID3 builds a decision tree top-down. At every node it picks the attribute with the highest information gain, where Gain(S, A) = Entropy(S) − Σ (|Sᵥ|/|S|)·Entropy(Sᵥ) and Entropy(S) = −Σ pᵢ log₂ pᵢ.

```text
ID3(S, Attributes, Class)
1. Create a node N.
2. If all examples in S have the same class C
      return N as a leaf labelled C.
3. If Attributes is empty
      return N as a leaf labelled with the majority class of S.
4. For every attribute A compute Gain(S, A).
5. Choose A* with the highest information gain; label N with A*.
6. For each value v of A*:
      Sv = examples of S with A* = v
      if Sv is empty: add a leaf with the majority class of S
      else: add the subtree ID3(Sv, Attributes - {A*}, Class)
7. Return N.
```

#### Significance of the confusion matrix generated by WEKA
The confusion matrix tabulates the actual class of the test records (rows) against the class predicted by the model (columns). For two classes:

| | Predicted yes | Predicted no |
| --- | --- | --- |
| Actual yes | TP | FN |
| Actual no | FP | TN |

It shows not only how many records were classified correctly (the diagonal) but also which kind of mistakes are made, and so it is the basis for the performance measures:
- **Accuracy** = (TP + TN)/total; **error rate** = 1 − accuracy.
- **TP rate (recall, sensitivity)** = TP/(TP + FN); **FP rate** = FP/(FP + TN).
- **Precision** = TP/(TP + FP); F-measure = 2·P·R/(P + R).

It is useful for imbalanced classes (accuracy alone can mislead), for comparing classifiers, and for judging whether a particular type of error (e.g. missing a patient) is acceptable. Off-diagonal cells show which classes are confused with each other.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. In ID3, if all examples at a node have the same class, what happens?
   A. The node is split further
   B. The node is deleted
   C. The node becomes a leaf with that class
   D. The node takes the parent's majority class
   Answer: C. Entropy zero means a pure leaf. (Key point: ID3 step)

2. In the WEKA confusion matrix, what do rows and columns represent?
   A. Rows are predicted; columns are actual
   B. Both are attributes
   C. Rows are training; columns are test records
   D. Rows are actual classes; columns are predicted classes
   Answer: D. Cell (i,j) counts actual class i predicted as j. (Key point: Confusion matrix layout)

3. How is accuracy computed from the matrix?
   A. (TP + TN)/total
   B. TP/(TP + FP)
   C. TP/(TP + FN)
   D. FP/(FP + TN)
   Answer: A. Accuracy is the diagonal over total. (Key point: Accuracy formula)

4. Why is the confusion matrix more informative than accuracy alone?
   A. It removes errors from the model
   B. It shows which kinds of mistakes are made, important for imbalanced classes
   C. It shows the tree structure
   D. It computes entropy
   Answer: B. Accuracy can mislead with imbalanced classes. (Key point: Usefulness)

## Related in Data Mining

- [ID3 and TP rate, FP rate, precision](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aid3-weka-tp-fp-precision) — Data Mining, chapter 3
- [ID3 steps, overfitting, TP/FP/recall](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aid3-steps-overfitting-recall) — Data Mining, chapter 3
- [Confusion matrix and performance measures](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aconfusion-matrix-measures) — Data Mining, chapter 3
- [Measures from confusion matrix (152, 130)](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aconfusion-matrix-152-130) — Data Mining, chapter 3
- [Measures from cancer confusion matrix](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aconfusion-matrix-cancer) — Data Mining, chapter 3
- [ID3 algorithm and why greedy](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aid3-greedy) — Data Mining, chapter 3

## Related in other subjects

- [Significance of arrays; multiply 3×3 matrices](https://www.bctengineer.com/question.md?key=semester_1%2Fcomputer_programming%3Aarray-significance-multiply-matrices) — Computer Programming, chapter 7
- [Stiffness matrix: frame, BMD](https://www.bctengineer.com/question.md?key=semester_5%2Ftheory_of_structures_ii%3Astiffness-frame-2knm-10kn-20knm) — Theory of Structures II, chapter 3
- [Significance of circular linked list](https://www.bctengineer.com/question.md?key=semester_4%2Fdata_structure_and_algorithm%3Acircular-linked-list-significance) — Data Structure and Algorithm, chapter 4
- [Big-Oh significance and limitation](https://www.bctengineer.com/question.md?key=semester_4%2Fdata_structure_and_algorithm%3Abig-oh-significance-limitation) — Data Structure and Algorithm, chapter 9