1
$\begingroup$

I have the following dataset:

        node        bc cluster
1    russian  0.457039       1
48       man  0.286875       1
155    woman  0.129939       0
3        bit  0.092721       0
5      write  0.065424       0
98       age  0.064347       0
97     escap  0.062675       0
74      game  0.062606       0

Then I perform kMeans clustering by bc value to separate the nodes into two different groups. Right now with the code below I get the result above (the clustering result is in the cluster column).

    bc_df = pd.DataFrame({"node": bc_nodes, "bc": bc_values})
    bc_df = bc_df.sort_values("bc", ascending=False)
    km = KMeans(n_clusters=2).fit(bc_df[['bc']])
    bc_df.loc[:,'cluster'] = km.labels_
    print(bc_df.head(8))

Which is pretty good, but I would like it to work slightly differently and to select the first 4 nodes into the first cluster and then the other ones in the 2nd one, because they are more similar to each other.

Can I do some adjustment to kMeans or maybe you know another algorithm in sklearn that can do that?

Improve this question
$\endgroup$
7
  • $\begingroup$ If you want to separate 2 classes (clusters) by 1 numerical feature, maybe setting a threahould with otsu would wield better results $\endgroup$
    – Pedro Henrique Monforte
    Apr 2, 2019 at 22:00
  • $\begingroup$ @PedroHenriqueMonforte what is otsu and how do I set a threshold with it? It wouldn't be a numerical feature, I need the algorithm to know where to separate. $\endgroup$
    – Aerodynamika
    Apr 2, 2019 at 22:01
  • 1
    $\begingroup$ So, otsu is a threshoulding algorithm that finds the value that minimizes the intra-class variance and maximazes the inter-class variance. Sorry if that is not clear, never defined otsu's algorithm in English. It is common in image processing and it is kind of a clustering algorithm. $\endgroup$
    – Pedro Henrique Monforte
    Apr 2, 2019 at 22:06
  • 1
    $\begingroup$ Give me a minute to find a implementation for you $\endgroup$
    – Pedro Henrique Monforte
    Apr 2, 2019 at 22:07
  • 1
    $\begingroup$ Please don't call-post duplicates: stackoverflow.com/q/55476863/1060350 $\endgroup$
    – Has QUIT--Anony-Mousse
    Apr 2, 2019 at 22:14

1 Answer 1

Reset to default
1
$\begingroup$

Welcome, Dmitry!

A very first red alarm ... If you are doing clustering, then let it tell you who is similar to who! If you already know your clusters, then why are you doing clustering indeed?!

And about the algorithms: Yes there are many more, for example, Spectral Clustering which makes use of different similarity measures (via Gaussian Kernel or K-NN connectivity matrix) to find clusters or DBSCAN which is based on similarity according to the density of data points.

If you needed more help please drop a comment here!

Good Luck!

Improve this answer
$\endgroup$
2
  • $\begingroup$ Great comment :) but I don't know the clusters, rather, I know the principles how I want my clusters to be selected. Do you know how I could apply these other algorithms you mentioned to this problem? Thanks! $\endgroup$
    – Aerodynamika
    Apr 2, 2019 at 14:36
  • $\begingroup$ If you mean programming-wise, there are as simple as kmeans. All of them have a fit method which gets your data and outputs the model. Documentation will defenitely help. Everything is exactly the same e.g. "clustering = DBSCAN(eps=3, min_samples=2).fit(bc_df[['bc']])" $\endgroup$
    – Kasra Manshaei
    Apr 2, 2019 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.