We are trying a DBSCAN clustering model on our 30,000 samples with 15 features each. We tuned the epsilon parameter small enough to make sure the radius of the clustering circle is small while it does clustering. We expected to have small clusters with high density. After clustering and parameters tuning, we used t-SNE to plot the clustering results in 2 dimensional space, we found that we have small clusters like cluster 2,3,4,5 with high density as expected while large clusters like cluster 0,1 scattered loosely as unexpected. obviously, cluster 0, 1 looks very strange to us.

Is there something we are not aware of, is it something wrong with clustering algorithm?

Or it's something about t-SNE, when it plots the clustering results in lower dimension, it might look like cluster 0, 1 are very scattered, actually they are more close and tensed in higher dimensional space?

  • cluster 0: orange
  • cluster 1: blue
  • cluster 2: red
  • cluster 3: green
  • cluster 4: light yellow
  • cluster 5: purple

30,000 samples, 6 clusters case:

enter image description here

30,000 samples, 8 clusters case:

enter image description here

  • 2
    $\begingroup$ tSNE does not preserve density, so it doesn't preserve clusters, see stats.stackexchange.com/a/264647/7828 $\endgroup$ – Has QUIT--Anony-Mousse Oct 27 '17 at 21:08
  • $\begingroup$ @Anony-Mousse Thanks for the reference, if that's the case, do you have any suggestions if I'd like to plot the cluster results with the density preserved as best as possible. $\endgroup$ – Shengjie Oct 28 '17 at 2:29
  • $\begingroup$ MDS may be better suited, because it approx. preserves distance, hence density. $\endgroup$ – Has QUIT--Anony-Mousse Oct 28 '17 at 12:43
  • $\begingroup$ @Anony-Mousse MDS stands for? $\endgroup$ – Shengjie Oct 30 '17 at 1:42
  • $\begingroup$ Multidimensional Scaling. A standard visualization technique. $\endgroup$ – Has QUIT--Anony-Mousse Oct 30 '17 at 8:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.