--- title: "ROC curve for model selection" subject: "Data Mining" chapter: 3 key: "semester_7/data_mining:roc-curve" source: https://www.bctengineer.com/q/semester_7/data_mining/roc-curve site: BCT Engineering --- # ROC curve for model selection Data Mining · Chapter 3: Classification - Asked in: year not given - Video: [#1. How to plot ROC Curve | Area Under Curve False Positive Rate vs True Positive Rate Mahesh Huddar](https://www.youtube.com/watch?v=0awZbJTwgMM) — Mahesh Huddar - Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14 - Page: https://www.bctengineer.com/q/semester_7/data_mining/roc-curve - Whole chapter: https://www.bctengineer.com/md/semester_7/data_mining/chapter-3 ## Question Explain the role of Receiver Operating Characteristics (ROC) Curve for classifier model selection. ## Short answer An ROC curve plots TPR vs FPR as threshold varies. A curve closer to the top-left corner and higher AUC indicate better classifier performance, helping compare models across all thresholds. ## Answer A **Receiver Operating Characteristic (ROC) curve** is a graph that shows the trade-off between the **true positive rate** and the **false positive rate** of a classifier as its decision threshold is varied. It is used to compare classification models visually. - Y-axis: **TPR** (sensitivity) = TP/(TP + FN) - X-axis: **FPR** (1 − specificity) = FP/(FP + TN) #### Construction Rank the test tuples by the classifier's probability (or score) for the positive class, in decreasing order. Starting from (0, 0), move through the list: for each tuple that is truly positive move up, and for each negative move right (by the corresponding steps). Each threshold gives one point (FPR, TPR). ```text TPR 1.0 | ,-----------* perfect: (0,1) | ,-' model M1 | ,' ..'' model M2 | / ..'' . diagonal = random guess | /.' 0.0 +-----------------------> FPR 0 1.0 ``` #### Role in model selection - The **diagonal** line from (0, 0) to (1, 1) is a random classifier; a better model has a curve closer to the **top-left corner** (0, 1). - The **area under the curve (AUC)** summarises accuracy: 0.5 = random, 1.0 = perfect. The model with the larger AUC is preferred. Rule of thumb: 0.9-1.0 excellent, 0.8-0.9 good, 0.7-0.8 fair. - The shape of the curve helps in choosing a **threshold** matching the cost of errors (e.g. high TPR for cancer screening). - It is independent of class distribution and costs, so is useful with imbalanced data. - Two models can be compared across all thresholds, not only at one operating point. ## Check yourself The answer's key points as multiple-choice questions, with the correct option and why. 1. What do the ROC axes show? A. Y: TPR, X: FPR B. Y: FPR, X: TPR C. Y: precision, X: recall D. Y: accuracy, X: threshold Answer: A. ROC plots TPR against FPR. (Key point: Axes) 2. What does the diagonal line represent? A. A perfect classifier B. A random classifier C. The best threshold D. Zero error Answer: B. Points on the diagonal have TPR = FPR. (Key point: Diagonal) 3. Which model is preferred by AUC? A. The one with smaller area B. The one closest to the diagonal C. The one with larger area under the curve D. The one with the longest curve Answer: C. AUC 1.0 is perfect and 0.5 random. (Key point: AUC) 4. Why is ROC useful with imbalanced data? A. It ignores positives B. It uses accuracy C. It needs equal classes D. It is independent of class distribution and costs Answer: D. TPR and FPR are each normalised within their own class. (Key point: Imbalanced data) ## Related in Data Mining - [Confusion matrix and performance measures](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aconfusion-matrix-measures) — Data Mining, chapter 3 - [Inverse relation of precision and recall](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aprecision-recall-inverse) — Data Mining, chapter 3 - [Measures from confusion matrix (152, 130)](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aconfusion-matrix-152-130) — Data Mining, chapter 3 - [Measures from cancer confusion matrix](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aconfusion-matrix-cancer) — Data Mining, chapter 3 - [Sequential covering for rule induction](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Asequential-covering) — Data Mining, chapter 3 ## Related in other subjects - [HRC fuse operating characteristics and applications](https://www.bctengineer.com/question.md?key=semester_6%2Fswitchgear_and_protection%3Ahrc-fuse-operating-characteristics) — Switchgear and Protection, chapter 3 - [Torque-slip curve: starting, maximum and operating region](https://www.bctengineer.com/question.md?key=semester_4%2Felectrical_machines_i%3Aim-torque-slip-starting-max-operating) — Electrical Machines I, chapter 5 - [Role and characteristics of organizations](https://www.bctengineer.com/question.md?key=semester_7%2Forganization_and_management%3Aorg-role-characteristics) — Organization and Management, chapter 1 - [Motor selection characteristics; rheostatic and regenerative braking](https://www.bctengineer.com/question.md?key=semester_7%2Futilization_of_electrical_energy%3Amotor-selection-characteristics-braking) — Utilization of Electrical Energy, chapter 2