3
$\begingroup$

K-modes algorithm is available here

I want to do clustering of my binary dataset. I need to specify the number of clusters that I need as an output:

KModes (n_clusters, init, n_init, verbose)

My dataset contains 1000 lines and 1000 rows, I want to calculate the distance between my clusters in order to know the exact number of cluster that I need to choose. I don't know how to compare between them. Probably I want to use the hamming distance because it is the most suitable distance to compare between binary data.

Any recommendations that you can give are welcomed.

$\endgroup$
8
$\begingroup$

The are some techniques to choose the number of clusters K.

The most common ones are The Elbow Method and The Silhouette Method.

Elbow Method

In this method, you calculate a score function with different values for K. You can use the Hamming distance like you proposed, or other scores, like dispersion.

Then, you plot them and where the function creates "an elbow" you choose the value for K.

enter image description here

Silhouette Method

This method measure the distance from points in one cluster to the other clusters. Then visually you have silhouette plots that let you choose K.

Observe:

K=2, silhouette of similar heights but with different sizes. So, potential candidate. enter image description here

K=3, silhouettes of different heights. So, bad candidate. enter image description here

K=4, silhouette of similar heights and sizes. Best candidate. enter image description here

| improve this answer | |
$\endgroup$
  • $\begingroup$ very thorough answer! $\endgroup$ – Valentin Calomme Dec 9 '19 at 9:28
  • $\begingroup$ Glad to hear it :) $\endgroup$ – Bruno Lubascher Dec 10 '19 at 17:36
  • $\begingroup$ So the same techniques that apply to Kmeans, also work on Kmodes? $\endgroup$ – Carlos Mougan Jan 5 at 19:04
  • $\begingroup$ @CarlosMougan yes $\endgroup$ – Islacine Feb 23 at 17:47
0
$\begingroup$

The KMeans algo is pretty slick, but it's a bit primitive compared to other algos out there. If you are interested in having a clustering algo automatically figure out the number of clusters for you, so you don't have to, you can use Affinity Propagation.

from sklearn.cluster import AffinityPropagation
from sklearn import metrics
from sklearn.datasets import make_blobs

# #############################################################################
# Generate sample data
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import numpy as np
#%matplotlib inline
from sklearn import datasets#Iris Dataset
iris = datasets.load_iris()
X = iris.data

# #############################################################################
# Compute Affinity Propagation
af = AffinityPropagation(preference=-50).fit(X)
cluster_centers_indices = af.cluster_centers_indices_
labels = af.labels_

n_clusters_ = len(cluster_centers_indices)

print('Estimated number of clusters: %d' % n_clusters_)


# #############################################################################
# Plot result
import matplotlib.pyplot as plt
from itertools import cycle

plt.close('all')
plt.figure(1)
plt.clf()

colors = cycle('bgrcmykbgrcmykbgrcmykbgrcmyk')
for k, col in zip(range(n_clusters_), colors):
    class_members = labels == k
    cluster_center = X[cluster_centers_indices[k]]
    plt.plot(X[class_members, 0], X[class_members, 1], col + '.')
    plt.plot(cluster_center[0], cluster_center[1], 'o', markerfacecolor=col,
             markeredgecolor='k', markersize=14)
    for x in X[class_members]:
        plt.plot([cluster_center[0], x[0]], [cluster_center[1], x[1]], col)

plt.title('Estimated number of clusters: %d' % n_clusters_)
plt.show()

Output: Estimated number of clusters: 3

enter image description here

https://scikit-learn.org/stable/auto_examples/cluster/plot_affinity_propagation.html#sphx-glr-auto-examples-cluster-plot-affinity-propagation-py

https://scikit-learn.org/stable/modules/clustering.html

| improve this answer | |
$\endgroup$
  • $\begingroup$ are you going to say how any of this relates to kmodes? $\endgroup$ – baxx Oct 31 at 17:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.