--- title: "Inductive learning and ID3" subject: "Artificial Intelligence" chapter: 6 key: "semester_6/artificial_intelligence:inductive-learning-id3" source: https://www.bctengineer.com/q/semester_6/artificial_intelligence/inductive-learning-id3 site: BCT Engineering --- # Inductive learning and ID3 Artificial Intelligence · Chapter 6: Machine Learning - Asked in: 2074 Bhadra, 2+6 marks - Video: [ID3 Decision Tree Learning Inductive Bias | Inductive bias of ID3 | Occam's razor ID3 Mahesh Huddar](https://www.youtube.com/watch?v=SVwFJZeWdtg) — Mahesh Huddar - Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14 - Page: https://www.bctengineer.com/q/semester_6/artificial_intelligence/inductive-learning-id3 - Whole chapter: https://www.bctengineer.com/md/semester_6/artificial_intelligence/chapter-6 ## Question Define inductive learning. Explain in detail about the ID3 process with suitable example. ## Short answer Inductive learning derives general rules from specific examples. ID3 builds a decision tree by selecting the attribute with highest information gain at each node. For the Play Tennis example, Outlook is the root split into Sunny (Humidity), Overcast (Yes), Rain (Wind). ## Answer **Inductive learning** is learning a general rule or concept from specific training examples (specific to general). Given examples with attribute values and class labels, the learner builds a hypothesis (such as a decision tree) that classifies new, unseen cases. #### ID3 (Iterative Dichotomiser 3, Quinlan) ID3 builds a decision tree top-down; at each node it picks the attribute with the highest **information gain**. - Entropy: E(S) = −Σ p_i log₂ p_i - Gain(S, A) = E(S) − Σ (|S_v| / |S|) · E(S_v) #### Steps 1. If all examples in S belong to one class, make a leaf with that class. 2. If no attributes remain, make a leaf with the majority class. 3. Otherwise compute Gain for each attribute; choose the best A. 4. Create a node for A; split S by the values of A. 5. Recurse on each subset with A removed. #### Example: Play Tennis (14 examples: 9 Yes, 5 No) E(S) = −(9/14)log₂(9/14) − (5/14)log₂(5/14) = **0.940** | Attribute | Subsets (Yes/No) | Gain | |---|---|---| | Outlook | Sunny 2/3, Overcast 4/0, Rain 3/2 | **0.247** | | Temperature | Hot 2/2, Mild 4/2, Cool 3/1 | 0.029 | | Humidity | High 3/4, Normal 6/1 | 0.152 | | Wind | Weak 6/2, Strong 3/3 | 0.048 | Outlook has the highest gain, so it is the root. - Overcast: all Yes, so leaf = Yes. - Sunny (E = 0.971): gains: Humidity 0.971, Temperature 0.571, Wind 0.020. Choose Humidity: High gives No, Normal gives Yes. - Rain (E = 0.971): Wind separates perfectly: Weak gives Yes, Strong gives No. ```text Outlook / | \ Sunny Overcast Rain | | | Humidity Yes Wind / \ / \ High Normal Weak Strong No Yes Yes No ``` (Gains were computed by program.) The tree gives rules such as: IF Outlook = Sunny AND Humidity = Normal THEN Play = Yes. **Strengths:** simple, readable rules, fast. **Weaknesses:** overfits noisy data (needs pruning), prefers attributes with many values. ## Check yourself The answer's key points as multiple-choice questions, with the correct option and why. 1. How does ID3 choose the attribute at each node? A. The attribute with the most values B. The attribute with the lowest entropy gain C. Random choice D. The attribute with the highest information gain Answer: D. Gain(S,A) = E(S) - sum |Sv|/|S| E(Sv). (Key point: Attribute choice) 2. In Play Tennis, which attribute is the root and why? A. Outlook, with the highest gain (0.247) B. Humidity, with gain 0.152 C. Wind, with gain 0.048 D. Temperature, with gain 0.029 Answer: A. Outlook has the largest gain. (Key point: Root) 3. What is the entropy of 9 Yes and 5 No? A. 0.5 B. 0.940 C. 1.0 D. 0.247 Answer: B. -(9/14)log2(9/14) - (5/14)log2(5/14) = 0.940. (Key point: Entropy) 4. What happens at the Overcast branch? A. It splits on Wind B. It splits on Humidity C. All examples are Yes, so it becomes a leaf with Yes D. It is pruned with No Answer: C. A pure subset becomes a leaf. (Key point: Overcast branch) ## Related in Artificial Intelligence - [Analogy vs inductive learning](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Aanalogy-vs-inductive-learning) — Artificial Intelligence, chapter 6 - [Machine learning; Boltzmann machines](https://www.bctengineer.com/question.md?key=semester_6%2Fartificial_intelligence%3Aml-boltzmann-machine) — Artificial Intelligence, chapter 6 ## Related in other subjects - [Process of pipe laying and jointing](https://www.bctengineer.com/question.md?key=semester_5%2Fwater_supply_engineering%3Apipe-laying-jointing) — Water Supply Engineering, chapter 8, asked 7× - [JPEG compression process steps](https://www.bctengineer.com/question.md?key=semester_8%2Fmultimedia_system%3Ajpeg-compression-steps) — Multimedia System, chapter 5, asked 3× - [Process control block](https://www.bctengineer.com/question.md?key=semester_6%2Foperating_system%3Apcb-explain) — Operating System, chapter 2 - [MapReduce vs Hadoop; IR process](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Amapreduce-vs-hadoop-ir) — Information System, chapter 8