--- title: "Likelihood and distance; K-means drawbacks" subject: "Data Mining" chapter: 3 key: "semester_7/data_mining:likelihood-distance-clustering-kmeans-drawbacks" source: https://www.bctengineer.com/q/semester_7/data_mining/likelihood-distance-clustering-kmeans-drawbacks site: BCT Engineering --- # Likelihood and distance; K-means drawbacks Data Mining · Chapter 3: Classification - Asked in: year not given, 4+2+3 marks - Video: [StatQuest: K-means clustering](https://www.youtube.com/watch?v=4b5d3muPQmA) — StatQuest with Josh Starmer - Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14 - Page: https://www.bctengineer.com/q/semester_7/data_mining/likelihood-distance-clustering-kmeans-drawbacks - Whole chapter: https://www.bctengineer.com/md/semester_7/data_mining/chapter-3 ## Question Define Likelihood and Distance. Do we need likelihood and distance in clustering? Explain the drawbacks of K-means clustering algorithm. ## Short answer Likelihood is the probability of an object under a class or model; distance measures dissimilarity in attribute space. Both are needed: distance assigns objects to nearest centroids or clusters, while likelihood is maximised in model-based clustering. K-means drawbacks: k must be fixed, initial centroids affect results, outliers and noise hurt it, it finds only spherical clusters, and it needs numeric data. ## Answer #### Likelihood and distance - **Likelihood** is the probability that an object belongs to a given class or model, e.g. P(X | Cᵢ). The class with the highest likelihood is chosen. - **Distance** is a numeric measure of how far two objects are in attribute space, e.g. Euclidean d(x, y) = √Σ(xᵢ − yᵢ)². Principle: objects that are close to each other are alike. #### Are they needed in clustering? Yes. Clustering groups objects so that those in a cluster are similar and those in different clusters are dissimilar, and **distance** (or similarity) is how this "similarity" is measured. K-means assigns each object to the nearest centroid by distance; hierarchical clustering merges the closest clusters. In model-based clustering (e.g. EM with Gaussian mixtures) the **likelihood** of the data under the model is maximised and each object goes to the cluster with the highest likelihood. Without them no object could be placed in a cluster. #### Drawbacks of K-means 1. The number of clusters **k must be given** in advance. 2. Results depend on the **initial centroids**; it may reach a local optimum, so it is run several times. 3. **Sensitive to noise and outliers**, because the mean is affected by extreme values. 4. Works only where a **mean** can be defined (numeric data); categorical data needs k-modes. 5. Finds only **spherical, similar-size, similar-density** clusters, not arbitrary shapes. 6. Hard to use with very high dimensions, and cost O(nkt) is large for huge data. 7. Empty clusters can occur. ## Check yourself The answer's key points as multiple-choice questions, with the correct option and why. 1. Why is distance needed in clustering? A. It chooses class labels from training data B. It removes outliers automatically C. It measures similarity so objects can be assigned to the nearest centroid/closest cluster D. It sets k Answer: C. Without distance no object could be placed in a cluster. (Key point: Need in clustering) 2. In model-based clustering such as EM, how is an object assigned? A. To the nearest centroid by Euclidean distance B. To the largest cluster C. Randomly D. To the cluster with the highest likelihood Answer: D. EM maximises the likelihood of data under the model. (Key point: Model-based clustering) 3. Why is k-means sensitive to outliers? A. The mean is affected by extreme values B. It uses the median C. It ignores distance D. It uses class labels Answer: A. Extreme values shift the centroid. (Key point: Outliers) 4. What problem arises from k-means' dependence on initial centroids? A. It always finds the global optimum B. It may converge to a local optimum, so it is run several times C. It cannot run on numeric data D. It needs no k Answer: B. Different starts give different results. (Key point: Initial centroids) ## Related in Data Mining - [Need of clustering; hierarchical vs partitioning; K-means](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Acluster-need-hier-vs-partition-kmeans) — Data Mining, chapter 5 - [Short note: K-means clustering](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Akmeans-note) — Data Mining, chapter 5 - [Clustering, need and qualities](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Aclustering-define-qualities) — Data Mining, chapter 5 - [Short note: Likelihood and distance](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Alikelihood-distance-note) — Data Mining, chapter 3 - [DM techniques; likelihood and distance principle](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Adm-techniques-close-data-alike) — Data Mining, chapter 3 - [Cluster analysis; K-means on height/width](https://www.bctengineer.com/question.md?key=semester_7%2Fdata_mining%3Akmeans-height-width) — Data Mining, chapter 5 ## Related in other subjects - [Why do we need DFT](https://www.bctengineer.com/question.md?key=semester_7%2Fdigital_signal_analysis_and_processing%3Awhy-need-dft) — Digital Signal Analysis and Processing, chapter 7, asked 6× - [Why do we need FFT](https://www.bctengineer.com/question.md?key=semester_7%2Fdigital_signal_analysis_and_processing%3Awhy-need-fft) — Digital Signal Analysis and Processing, chapter 7, asked 2× - [Election algorithm need and example](https://www.bctengineer.com/question.md?key=semester_7%2Fdistributed_system%3Aelection-scenario) — Distributed System, chapter 6, asked 2× - [Algorithm and flowchart: distance between points](https://www.bctengineer.com/question.md?key=semester_1%2Fcomputer_programming%3Aalgo-flowchart-distance) — Computer Programming, chapter 2