2
$\begingroup$

Hdbscan is an excellent technique to find the "optimal" number of clusters within your data when you have little a priori idea how many clusters should exist. This makes the method great for exploratory analysis:

http://hdbscan.readthedocs.io/en/latest/comparing_clustering_algorithms.html

Here's my problem: All results using hdbscan with the python implement in the link above rely on the crucial min_cluster_size

http://hdbscan.readthedocs.io/en/latest/parameter_selection.html

If users have a priori little idea how many clusters best fit the data, what is the correct approach above? Isn't there a metric one uses to decide what the optimal number of clusters is?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Optimal in which sense?

The crucial thing with clustering is that there is no optimal solution. Different solutions tell you a different part of the story. And to be able to get different views, you will need parameters. It is a exploratory technique.

Various attempts at defining "optimal" solutions have failed for practical use, just think of k-means.

Improve this answer
$\endgroup$
6
  • $\begingroup$ By "optimal", I mean certain clusters fit the data better than others. $\endgroup$
    – ShanZhengYang
    Commented Feb 7, 2017 at 21:36
  • $\begingroup$ Define "fit the data". min_cluster_size=1 supposedly is the "best fit" yet subjectively usually much worse. There exists no objective notion of "best" that people find useful... $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Feb 7, 2017 at 22:03
  • $\begingroup$ So what's the point of clustering? $\endgroup$
    – ShanZhengYang
    Commented Feb 8, 2017 at 17:09
  • $\begingroup$ Tool to give you ideas how to look at your data. Nothing automatic, too unreliable. $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Feb 8, 2017 at 20:38
  • $\begingroup$ We're off the beaten track, but after getting a set of clusters...what am I supposed to infer with these? Investigate the properties of clusters? The consequences of the techniques are slightly strange $\endgroup$
    – ShanZhengYang
    Commented Feb 9, 2017 at 20:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.