0
$\begingroup$

I am trying to do clustering in my dataset which has 4 numerical fields. Please find the file attached : http://www.filedropper.com/example_3

I tried with this code:

from sklearn.cluster import KMeans
kmeans = KMeans(n_clusters=2, random_state=0, max_iter = 300).fit(dffinal)

I know that there are 2 classes in this example, that was the reason i tried with 2 clusters. Out of 4200 rows, first 3196 rows belong to a class and the remaining rows belong to an another class.

But when i do clustering, cluster labels are randomly assigned, and accuracy is less than 10%. Just wondering whether my features are not good enough for clustering or should I try with some other clustering algorithm.

Any help would be appreciated. Thanks.

$\endgroup$
  • 2
    $\begingroup$ Are you trying to cluster or classify your data? It sounds like you really want a classification algorithm. $\endgroup$ – Ryan Dec 23 '16 at 18:48
  • $\begingroup$ As @Ryan mentioned, you must be looking at classification not clustering. Got into same cluster does not always mean that data points belong to same class. $\endgroup$ – Kiritee Gak Dec 23 '16 at 20:48
2
$\begingroup$

You forgot to preprocess your data.

K-means is really sensitive to scale and outliers.

Also: clustering is not classification.

It may well be that e.g. one of your classes has a dense subtype. Just because the clustering found something else than your labels (and it's not supposed to find the same, is it?) does not mean it failed. It just didn't classify the data the same way as your labels (but if that is what you wsnt, then you should have been using a classification method instead.)

$\endgroup$
1
$\begingroup$

I suspect that clustering this data will not be very productive. Just make a simple plot showing two variables at a time. Pairs of variables None of these seems to give evidence of natural clustering. One of these plots may give you a glimmer of hope: Count vs Duration3 Count vs Duration3 There is something going on here that some points have Count bigger than 10, others have Duration3 bigger than 10^6 and those two things never happen together. This might hint at a mixture of two groups, but I doubt that clustering is the way to get at that. Most of the points are in the puddle near the origin.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.