---
title: "Short note: Decision tree"
subject: "Data Mining"
chapter: 3
key: "semester_7/data_mining:decision-tree-note"
source: https://www.bctengineer.com/q/semester_7/data_mining/decision-tree-note
site: BCT Engineering
---

# Short note: Decision tree

Data Mining · Chapter 3: Classification

- Asked in: 2065 Baisakh
- Video: [Lec-9: Introduction to Decision Tree 🌲 with Real life examples](https://www.youtube.com/watch?v=mvveVcbHynE) — Gate Smashers
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_7/data_mining/decision-tree-note
- Whole chapter: https://www.bctengineer.com/md/semester_7/data_mining/chapter-3

## Question

Write a short note on Decision Tree.

## Short answer

A decision tree classifies by testing attributes at internal nodes, branching on outcomes, and assigning leaf class labels. It is built top-down by splitting on the best attribute (information gain, gain ratio, Gini) and pruned to avoid overfitting. Key algorithms include ID3, C4.5, CART.

## Answer

A **decision tree** is a flowchart-like tree used for classification and prediction. Each **internal node** tests an attribute, each **branch** is an outcome of the test, and each **leaf node** holds a class label. The topmost node is the root.

#### How it works
To classify a tuple, start at the root, follow the branch matching the tuple's attribute value at each node, until a leaf is reached; its label is the predicted class.

```text
        [Age?]
      /   |    \
  young  middle  senior
    |      |       |
[Student?] YES  [Credit?]
  /   \          /    \
 no   yes     fair   excellent
 NO   YES     YES       NO
```

#### Construction
The tree is built top-down by recursive partitioning (greedy, divide and conquer). The best splitting attribute at each step is chosen using a measure such as **information gain** (ID3), **gain ratio** (C4.5) or **Gini index** (CART). Growing stops when all records of a node have the same class, no attributes remain, or no records are left.

#### Features
- Does not need domain knowledge or parameter setting; handles numeric and categorical data.
- Fast to build and to classify, with accuracy comparable to other methods.
- Easy to understand and convert into IF-THEN rules.
- May overfit noisy data, so **pruning** (pre- or post-pruning) is used.

Common algorithms: ID3, C4.5, CART.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. In a decision tree, what does each leaf node hold?
   A. A class label
   B. A test on an attribute
   C. An outcome of a test
   D. The splitting measure value
   Answer: A. Internal nodes test attributes, branches are outcomes, and leaves hold the class label. (Key point: Tree components)

2. How is a new tuple classified using a decision tree?
   A. Compute the distance to all stored tuples and take the majority class
   B. Start at the root and follow the branch matching its attribute value at each node until a leaf is reached
   C. Start at a leaf and move up to the root
   D. Average the class labels of all leaves
   Answer: B. The tuple follows the matching branches from the root to a leaf, whose label is the prediction. (Key point: Classifying a tuple)

3. Which pairing of algorithm and splitting measure is correct?
   A. ID3 - Gini index
   B. CART - information gain
   C. C4.5 - gain ratio
   D. C4.5 - Gini index
   Answer: C. ID3 uses information gain, C4.5 gain ratio and CART the Gini index. (Key point: Splitting measures)

4. Why is pruning used with decision trees?
   A. Trees cannot handle categorical data
   B. Trees are slow to classify
   C. Trees need domain knowledge to build
   D. Trees may overfit noisy data
   Answer: D. Pruning (pre or post) reduces overfitting to noise. (Key point: Overfitting)

## Related in Data Mining

- [Overfitting in ID3 and decision tree rules](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aid3-overfitting-rules) — Data Mining, chapter 3
- [Decision tree classifier and Hunt's algorithm](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Adecision-tree-hunt) — Data Mining, chapter 3
- [Decision tree advantages and disadvantages](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Adecision-tree-pros-cons) — Data Mining, chapter 3
- [Extracting rules from decision tree](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Arules-from-decision-tree) — Data Mining, chapter 3

## Related in other subjects

- [Explain decision tree analysis](https://www.bctengineer.com/question.md?key=semester_6%2Fengineering_economics%3Aexplain-decision-tree) — Engineering Economics, chapter 6, asked 5×
- [Decision tree for plant improvement alternatives](https://www.bctengineer.com/question.md?key=semester_6%2Fengineering_economics%3Adecision-tree-plant-alternatives) — Engineering Economics, chapter 6, asked 2×
- [Decision tree; infinite source blocking formulas](https://www.bctengineer.com/question.md?key=semester_8%2Ftelecommunications%3Adecision-tree-infinite-sources) — Telecommunications, chapter 6, asked 2×
- [Decision tree; finite source blocking formulas](https://www.bctengineer.com/question.md?key=semester_8%2Ftelecommunications%3Adecision-tree-finite-sources) — Telecommunications, chapter 6, asked 2×