2
$\begingroup$

I am trying to cluster the sample of Imagenet Dataset using K-Means clustering.

In this approach, I have used the below 2 approaches to get the optimal number of clusters.

  1. Elbow method
    enter image description here From the Graph it seems like the best number of clusters is from 6 to 10.

  2. Silhouette score

Cluster : 2 | Silhouette score : 0.036273542791604996
Cluster : 3 | Silhouette score : -0.00300691369920969
Cluster : 4 | Silhouette score : 0.0025101888459175825
Cluster : 5 | Silhouette score : -0.005924953147768974
Cluster : 6 | Silhouette score : -0.00808520708233118
Cluster : 7 | Silhouette score : -0.006091121584177017
Cluster : 8 | Silhouette score : -0.00549863139167428
Cluster : 9 | Silhouette score : -0.014739749021828175
Cluster : 10 | Silhouette score : -0.021131910383701324
Cluster : 11 | Silhouette score : -0.04057755321264267
Cluster : 12 | Silhouette score : -0.012825582176446915
Cluster : 13 | Silhouette score : -0.012340431101620197
Cluster : 14 | Silhouette score : -0.032936643809080124
Cluster : 15 | Silhouette score : -0.04154697805643082
Cluster : 16 | Silhouette score : -0.04323640838265419

Where as in Silhouette analysis it looks like only for cluster 4 it is showing better value. Rest of the clusters are seems like samples are wrongly assigned to wrong clusters.

In such cases, which metric needs to be considered?

Updates:
I have reduced the dimension of the features using PCA. Below are the updated graphs for Elbow and Silhouette analysis. However I do not see any improvement in the clustering. As per Silhouette the samples are not being assigned to the closest cluster.

Elbow
enter image description here

Silhouette
enter image description here

Updating the question with latest graphs, VGGNet has been replaced by Resnet50

enter image description here

Code snippet:

        for i in range(lower_range, upper_range):
            # For Elbow curve
            kmeans_cluster = KMeans(n_clusters = i)
            kmeans_cluster_fit = kmeans_cluster.fit(features_list_np)
            loss = kmeans_cluster_fit.inertia_
            error.append(kmeans_cluster_fit.inertia_)

            # For Silhoutte analysis
            preds = kmeans_cluster.fit_predict(features_list_np)
            score = silhouette_score(features_list_np, preds)
            score_list.append(score)

            sample_score = silhouette_samples(features_list_np, preds)
            sample_score_list.append(sample_score)

            print("Cluster : {} | Loss : {} | Silhoutte Score : {}".format(i, loss, score))

            zero_samples = 0
            positive_samples = 0
            negative_samples = 0

            for each_sample in sample_score:
                if each_sample == 0:
                    zero_samples += 1
                if each_sample > 0:
                    positive_samples += 1
                if each_sample < 0:
                    negative_samples += 1

            print("Cluster : {} | Silhouette sample distribution - Zero : {} | Positive : {} | Negative : {}".format(i, zero_samples, positive_samples, negative_samples))

            zero_list.append(zero_samples)
            pos_list.append(positive_samples)
            neg_list.append(negative_samples)

Also should the importance should be given to Silhouette Score or the Silhouette Samples. Because in some of the cases, Silhouette score is less but the number of samples having positive values are more.

Thank you,
KK

$\endgroup$
3
$\begingroup$

Neither, there is not enough discriminatory information in data (yet)

Dont squeeze the data until it tells you the truth. You can change the metric (malahobian distance for example) and the algo but you cant expect it to show miracles.

Using elbow method, as you increase the number of clusters it will always become more homogenous. You dont have a "kink" indicating optimal clustering number.

And with Silhouete (be careful those are averaged Silhoute score per cluster, in other words for k=4 you have 4 scores indicating wether points are lying inside/border/should- be-other-cluster that are averaged) you get that all of the points (in average) lie on the border of the clusters, without clear distinction of the clusters (thats what 0 means).

Advice Find better quantitative representation of data, new feature, reduce noise etc...

$\endgroup$
  • $\begingroup$ As mentioned in the below comment, I use pre-trained VGGNet to extract the features and using it for clustering. $\endgroup$ – deepguy Dec 23 '19 at 4:32
3
$\begingroup$

A Silhouette score close to 0 says the clustering is not reliable.

And the Elbow method is crap. On random data the curve would drop roughly like 1/(k-1); so it's largely undefined wh em they is an elbow and when not. In your case, what troubles me most is that the values appear to stagnate to a cake much larger than zero. Maybe there is an error in your clustering code? Your results look like random to me.

$\endgroup$
  • $\begingroup$ Thanks for the input. I am using samples of class from the Imagenet dataset and using it to cluster the images. Images are passed through pre-trained VGGNet to extract the features. And K-Means cluster is applied on the extracted features. $\endgroup$ – deepguy Dec 23 '19 at 4:09
  • 1
    $\begingroup$ Negative Silhouette on average should be impossible with k-means (a few points may have negative values, but the majority must be assigned to the nearest cluster). You really should debug carefully, the result seems to be corrupted. $\endgroup$ – Has QUIT--Anony-Mousse Dec 23 '19 at 8:38
  • $\begingroup$ There maybe possibility that data samples are overlapped with each other ? Correct me if I am wrong. $\endgroup$ – deepguy Dec 23 '19 at 9:53
  • $\begingroup$ That would still give a Silhouette of at least 0. Negative values mean that objects are, on average, not assigned to their nearest center! $\endgroup$ – Has QUIT--Anony-Mousse Dec 24 '19 at 7:36
  • $\begingroup$ Thanks for making me understanding. I am trying to figure what I am doing wrong. I even tried with PCA, however it did not performance of the model. $\endgroup$ – deepguy Dec 26 '19 at 5:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.