How to evaluate the K-Modes Clusters?

Question

K-modes algorithm is available here

I want to do clustering of my binary dataset. I need to specify the number of clusters that I need as an output:

KModes (n_clusters, init, n_init, verbose)

My dataset contains 1000 lines and 1000 rows, I want to calculate the distance between my clusters in order to know the exact number of cluster that I need to choose. I don't know how to compare between them. Probably I want to use the hamming distance because it is the most suitable distance to compare between binary data.

Any recommendations that you can give are welcomed.

Answer 1

The are some techniques to choose the number of clusters K.

The most common ones are The Elbow Method and The Silhouette Method.

Elbow Method

In this method, you calculate a score function with different values for K. You can use the Hamming distance like you proposed, or other scores, like dispersion.

Then, you plot them and where the function creates "an elbow" you choose the value for K.

Silhouette Method

This method measure the distance from points in one cluster to the other clusters. Then visually you have silhouette plots that let you choose K.

Observe:

K=2, silhouette of similar heights but with different sizes. So, potential candidate.

K=3, silhouettes of different heights. So, bad candidate.

K=4, silhouette of similar heights and sizes. Best candidate.

Answer 2

The KMeans algo is pretty slick, but it's a bit primitive compared to other algos out there. If you are interested in having a clustering algo automatically figure out the number of clusters for you, so you don't have to, you can use Affinity Propagation.

from sklearn.cluster import AffinityPropagation
from sklearn import metrics
from sklearn.datasets import make_blobs

# #############################################################################
# Generate sample data
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import numpy as np
#%matplotlib inline
from sklearn import datasets#Iris Dataset
iris = datasets.load_iris()
X = iris.data

# #############################################################################
# Compute Affinity Propagation
af = AffinityPropagation(preference=-50).fit(X)
cluster_centers_indices = af.cluster_centers_indices_
labels = af.labels_

n_clusters_ = len(cluster_centers_indices)

print('Estimated number of clusters: %d' % n_clusters_)


# #############################################################################
# Plot result
import matplotlib.pyplot as plt
from itertools import cycle

plt.close('all')
plt.figure(1)
plt.clf()

colors = cycle('bgrcmykbgrcmykbgrcmykbgrcmyk')
for k, col in zip(range(n_clusters_), colors):
    class_members = labels == k
    cluster_center = X[cluster_centers_indices[k]]
    plt.plot(X[class_members, 0], X[class_members, 1], col + '.')
    plt.plot(cluster_center[0], cluster_center[1], 'o', markerfacecolor=col,
             markeredgecolor='k', markersize=14)
    for x in X[class_members]:
        plt.plot([cluster_center[0], x[0]], [cluster_center[1], x[1]], col)

plt.title('Estimated number of clusters: %d' % n_clusters_)
plt.show()

Output: Estimated number of clusters: 3

https://scikit-learn.org/stable/auto_examples/cluster/plot_affinity_propagation.html#sphx-glr-auto-examples-cluster-plot-affinity-propagation-py

https://scikit-learn.org/stable/modules/clustering.html