3
$\begingroup$

I have a dataframe with 2 columns of numerical values. I want to apply a clustering algorithm to put all the entries into the same group, which have a relatively small distance to the other entries. But which clustering algorithm can I use, although I do not know how many groups will be formed? It would be ideal if there is a parameter to determine the maximum distance allowed. And if there isn't such an algorithm, maybe it would be really helpful to come up with some intuitions, how such an algorithm can be implemented by myself. Thanks a lot!! :)

The data could look like this:

a,b
20,30
19,31
10,10
9,8
12,11
31,11
32,11
| improve this question | | | | |
$\endgroup$
  • $\begingroup$ What have you tried? $\endgroup$ – Akavall Jan 9 at 1:18
3
$\begingroup$

I'd suggest looking at hierarchical clustering:

  • It's simple so you could implement and tune your own version
  • It lets you decide at which level you want to stop grouping elements together, so you could have a maximum distance.

Be careful however that this approach can sometimes lead to unexpected/non-intuitive clusters.

| improve this answer | | | | |
$\endgroup$
  • $\begingroup$ thanks! I tried Agglomerative clustering, it seems to be working well! :) $\endgroup$ – Enyang Wang Jan 9 at 9:08
3
$\begingroup$

I would try DBSCAN algorithm first: fairly easy to tune (with, in particular, a notion of distance as you requested), and does not need to know the number of clusters.

There are a few other algorithms that can help you decide the number of clusters: Bayesian Gaussian Mixtures (see sklearn implementation) for instance, but it requires a bit more knowledge and work. There is also spectral clustering, but for this one, sklearn does not automatically find the number of clusters, so you will have to do create your own implementation and determine the number yourself, manually, by plotting the eigenvalues.

| improve this answer | | | | |
$\endgroup$
  • $\begingroup$ looks very interesting, i will try this too! :) $\endgroup$ – Enyang Wang Jan 9 at 9:11
1
$\begingroup$

You could use K-means clustering as well here with euclidian distance measure..

Why I am suggesting euclidian distance because you have all numeric data, if it was mixed then gover distance was better pick and similarly you could pick correct distance measure based on requirements.

Here you can get the optimal number of clusters by nbclust function in R.

| improve this answer | | | | |
$\endgroup$
1
$\begingroup$

Here is a couple more based on affinity matrix:

and one similiar to DBSCAN and potentially better --- optics

| improve this answer | | | | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.