Hdbscan is an excellent technique to find the "optimal" number of clusters within your data when you have little a priori idea how many clusters should exist. This makes the method great for exploratory analysis:


Here's my problem: All results using hdbscan with the python implement in the link above rely on the crucial min_cluster_size


If users have a priori little idea how many clusters best fit the data, what is the correct approach above? Isn't there a metric one uses to decide what the optimal number of clusters is?


Optimal in which sense?

The crucial thing with clustering is that there is no optimal solution. Different solutions tell you a different part of the story. And to be able to get different views, you will need parameters. It is a exploratory technique.

Various attempts at defining "optimal" solutions have failed for practical use, just think of k-means.

  • $\begingroup$ By "optimal", I mean certain clusters fit the data better than others. $\endgroup$ – ShanZhengYang Feb 7 '17 at 21:36
  • $\begingroup$ Define "fit the data". min_cluster_size=1 supposedly is the "best fit" yet subjectively usually much worse. There exists no objective notion of "best" that people find useful... $\endgroup$ – Anony-Mousse Feb 7 '17 at 22:03
  • $\begingroup$ So what's the point of clustering? $\endgroup$ – ShanZhengYang Feb 8 '17 at 17:09
  • $\begingroup$ Tool to give you ideas how to look at your data. Nothing automatic, too unreliable. $\endgroup$ – Anony-Mousse Feb 8 '17 at 20:38
  • $\begingroup$ We're off the beaten track, but after getting a set of clusters...what am I supposed to infer with these? Investigate the properties of clusters? The consequences of the techniques are slightly strange $\endgroup$ – ShanZhengYang Feb 9 '17 at 20:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.