--- title: "Buys computer: ID3 and Naive Bayes" subject: "Data Mining" chapter: 3 key: "semester_7/data_mining:buys-computer-id3-naive-bayes" source: https://www.bctengineer.com/q/semester_7/data_mining/buys-computer-id3-naive-bayes site: BCT Engineering --- # Buys computer: ID3 and Naive Bayes Data Mining · Chapter 3: Classification - Asked in: year not given - Video: [7. Solved Example Naive Bayes Classification Age Income Student Credit Rating Buys Computer Mahesh](https://www.youtube.com/watch?v=ztYAWF8tzLI) — Mahesh Huddar - Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14 - Page: https://www.bctengineer.com/q/semester_7/data_mining/buys-computer-id3-naive-bayes - Whole chapter: https://www.bctengineer.com/md/semester_7/data_mining/chapter-3 ## Question Consider following training data (Buys Computer Data) | SN | Age | income | student | credit_rating | class: buys_computer | | --- | --- | --- | --- | --- | --- | | 1 | Young | high | no | Fair | no | | 2 | Young | high | no | Excellent | no | | 3 | middle_aged | high | no | Fair | yes | | 4 | Senior | medium | no | Fair | yes | | 5 | Senior | low | yes | Fair | yes | | 6 | Senior | low | yes | Excellent | no | | 7 | middle_aged | low | yes | Excellent | yes | | 8 | Young | medium | no | Fair | no | | 9 | Young | low | yes | Fair | yes | | 10 | Senior | medium | yes | Fair | yes | | 11 | Young | medium | yes | Excellent | yes | | 12 | middle_aged | medium | no | Excellent | yes | | 13 | middle_aged | high | yes | Fair | yes | | 14 | Senior | medium | no | Excellent | no | a. Determine the root node of decision tree for above data set using ID3 algorithm. b. Draw the complete Decision Tree using ID3 algorithm and determine class for X={age = young, income = low, student = yes and credit rating = fair} c. Use Naïve Bayes Classifier to determine class for i. X={age = young, income = low, student = yes and credit rating = fair} ii. X={age = middle_aged, income = low and credit rating = excellent} iii. X={age = senior and credit rating = excellent} ## Short answer Root node is age (gain 0.247). ID3 tree: age=middle_aged → yes; age=young → student=yes → yes; age=senior → credit_rating=fair → yes. X (young,low,yes,fair) → yes. Naïve Bayes: (i) yes, (ii) yes, (iii) no. ## Answer Total tuples = 14: buys_computer = yes: 9, no: 5. #### a. Root node using ID3 Entropy(D) = −(9/14)log₂(9/14) − (5/14)log₂(5/14) = **0.940** | Attribute | Value (yes, no) | Entropy of values | Weighted entropy | Gain | | --- | --- | --- | --- | --- | | age | young (2, 3), middle (4, 0), senior (3, 2) | 0.971, 0, 0.971 | 5/14·0.971 + 0 + 5/14·0.971 = 0.694 | 0.940 − 0.694 = **0.247** | | income | high (2, 2), medium (4, 2), low (3, 1) | 1.0, 0.918, 0.811 | 0.911 | 0.029 | | student | no (3, 4), yes (6, 1) | 0.985, 0.592 | 0.788 | 0.152 | | credit_rating | fair (6, 2), excellent (3, 3) | 0.811, 1.0 | 0.892 | 0.048 | **Answer: age has the highest gain (0.247), so the root node is `age`.** #### b. Complete decision tree - **middle_aged** (4 yes, 0 no): pure, leaf = yes. - **young** (2 yes, 3 no; E = 0.971): gains: income 0.571, student 0.971, credit 0.020, so split on **student**: no → no (3 records), yes → yes (2 records). - **senior** (3 yes, 2 no): gains: income 0.020, student 0.020, credit 0.971, so split on **credit_rating**: fair → yes (3), excellent → no (2). ```text age? young / middle \ senior student? YES credit_rating? no/ \yes fair/ \excellent NO YES YES NO ``` X = (age = young, income = low, student = yes, credit = fair): age = young → student = yes → **buys_computer = yes**. #### c. Naïve Bayes P(yes) = 9/14 = 0.643, P(no) = 5/14 = 0.357. Attributes not given in X are left out. **i. X = (young, low, student = yes, fair)** - P(X | yes) = (2/9)(3/9)(6/9)(6/9) = 0.03292; × P(yes) = **0.02116** - P(X | no) = (3/5)(1/5)(1/5)(2/5) = 0.0096; × P(no) = **0.00343** - 0.02116 > 0.00343, so **class = yes** (probability about 86%). **ii. X = (middle_aged, low, credit = excellent)** - yes: (4/9)(3/9)(3/9)·(9/14) = **0.03175** - no: (0/5)(1/5)(3/5)·(5/14) = **0** (zero count for middle_aged with "no") - **Class = yes.** With Laplacian correction: no = (5/14)(1/8)(2/8)(4/7) = 0.0064; yes = (9/14)(5/12)(4/12)(4/11) = 0.0325; the result is still **yes**. **iii. X = (senior, credit = excellent)** - yes: (3/9)(3/9)(9/14) = 0.0714 - no: (2/5)(3/5)(5/14) = 0.0857 - 0.0857 > 0.0714, so **class = no** (probability about 55%). **Answer: (i) yes, (ii) yes, (iii) no.** ## Check yourself The answer's key points as multiple-choice questions, with the correct option and why. 1. Which attribute is the root by ID3 and its gain? A. age, gain 0.247 B. student, gain 0.152 C. income, gain 0.029 D. credit_rating, gain 0.048 Answer: A. Age has highest information gain. (Key point: Root node) 2. Which attribute splits the 'young' subset? A. income B. student C. credit_rating D. age again Answer: B. Student gain is 0.971, giving a pure split. (Key point: Young branch) 3. Class for (young, low, student=yes, fair) by tree? A. no B. cannot be determined C. yes D. depends on income Answer: C. young -> student=yes -> yes. (Key point: Tree class) 4. In c(ii), why does P(X|no) = 0 before correction? A. Credit excellent never occurs B. Income low never occurs C. P(no) is 0 D. No tuple has middle_aged with class no Answer: D. The count for middle_aged in class no is 0, so the product is 0. (Key point: Zero count) ## Related in Data Mining - [Play golf: ID3 and Naive Bayes](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aplay-golf-id3-naive-bayes) — Data Mining, chapter 3 - [Naive Bayes and Laplacian correction](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Anaive-bayes-laplace) — Data Mining, chapter 3 - [Naive Bayes advantages and disadvantages](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Anaive-bayes-pros-cons) — Data Mining, chapter 3 ## Related in other subjects - [Central tendency; middle 70% income, median](https://www.bctengineer.com/question.md?key=semester_5%2Fprobability_and_statistics%3Acentral-tendency-middle-70-income) — Probability and Statistics, chapter 1 - [Abstract Student class program](https://www.bctengineer.com/question.md?key=semester_3%2Fobject_oriented_programming%3Astudent-abstract-program) — Object Oriented Programming, chapter 7