Is there a real life meaning about KMeans error?

Question

I am trying to understand the meaning of error in sklearn KMeans.

In the context of house pricing prediction, the error linear regression could be considered as the money difference per square foot.

Is there a real life meaning about KMeans error?

Answer 1

The K-means Error gives you, what is known as, total intra-cluster variance.

Intra-cluster variance is the measure of how much are the points in a given cluster spread.

The following cluster will have high intra-cluster variance

In the image below, even though the number of points are same as that of the image above, the points are densely distributed and hence will have lower intra-cluster variance.

K-means Error is interested in the total of such individual cluster variances.

Suppose for a given data, if clustering 'A' forms clusters like the first image and clustering 'B' forms clusters like the second image, you will in most cases choose the second one.

Although this does not mean that the K-means Error is a perfect objective to optimize on to form clusters. But it pretty much catches the essence behind clustering.

---

Code used for cluster plot generation -

import numpy as np
from matplotlib import pyplot as plt

sparse_samples = np.random.multivariate_normal([0, 0], [[50000, 0], [0, 50000]], size=(1000))
plt.plot(sparse_samples[:, 0], sparse_samples[:, 1], 'b+')
axes = plt.gca()
axes.set_xlim(-1000, 1000)
axes.set_ylim(-1000, 1000)
plt.show()

dense_samples = np.random.multivariate_normal([0, 0], [[5000, 0], [0, 5000]], size=(1000))
plt.plot(dense_samples[:, 0], dense_samples[:, 1], 'r+')
axes = plt.gca()
axes.set_xlim(-1000, 1000)
axes.set_ylim(-1000, 1000)
plt.show()

In both cases, a 1000 datapoints from a Bivariate Normal Distribution are sampled and plotted . In the second case, the Covariance Matrix is changed to plot a denser cluster. np.random.multivariate_normal's documentation can be found here. Hope this helps!

Answer 2

The meaning of "error" in k-Means clustering is: how much discrepancy / loss of information would you get if you substitute the k centroids to the actual observations. In other words: how good the k centroids approximate your data.

There are several ways you can measure this "error". Usually the percentage of variance explained or the within cluster sum of errors are employed, but the choice is huge. Even a more banal Euclidean distance could work.