Skip to main content

Likelihood and distance; K-means drawbacks

Short answer

Likelihood is the probability of an object under a class or model; distance measures dissimilarity in attribute space. Both are needed: distance assigns objects to nearest centroids or clusters, while likelihood is maximised in model-based clustering. K-means drawbacks: k must be fixed, initial centroids affect results, outliers and noise hurt it, it finds only spherical clusters, and it needs numeric data.

Define Likelihood and Distance. Do we need likelihood and distance in clustering? Explain the drawbacks of K-means clustering algorithm.

Likelihood and distance

  • Likelihood is the probability that an object belongs to a given class or model, e.g. P(X | Cᵢ). The class with the highest likelihood is chosen.
  • Distance is a numeric measure of how far two objects are in attribute space, e.g. Euclidean d(x, y) = √Σ(xᵢ − yᵢ)². Principle: objects that are close to each other are alike.

Are they needed in clustering?

Yes. Clustering groups objects so that those in a cluster are similar and those in different clusters are dissimilar, and distance (or similarity) is how this "similarity" is measured. K-means assigns each object to the nearest centroid by distance; hierarchical clustering merges the closest clusters. In model-based clustering (e.g. EM with Gaussian mixtures) the likelihood of the data under the model is maximised and each object goes to the cluster with the highest likelihood. Without them no object could be placed in a cluster.

Drawbacks of K-means

  1. The number of clusters k must be given in advance.
  2. Results depend on the initial centroids; it may reach a local optimum, so it is run several times.
  3. Sensitive to noise and outliers, because the mean is affected by extreme values.
  4. Works only where a mean can be defined (numeric data); categorical data needs k-modes.
  5. Finds only spherical, similar-size, similar-density clusters, not arbitrary shapes.
  6. Hard to use with very high dimensions, and cost O(nkt) is large for huge data.
  7. Empty clusters can occur.

✓ Answer checked by Claude Opus 5.5 (high effort)

Related questions

All Data Mining chapter 3 questions →