1
$\begingroup$

I have a big dataset with nearly 200 features. However, I do not have class labels for these data. I want to divide these data into two classes based on these features. I know, when we do not have class labels we have to use some clustering method. However, since I do not have any labels, I am just wondering how to measure the accuracy of the models.

Please let me know the most suitable approach that I should follow?

I am happy to provide more details about my featureset if needed :)

$\endgroup$
1
$\begingroup$

As you know, clustering is Unsupervised learning algorithm. Since you don't know the number of clusters, it becomes hard to find the best possible separation (number of clusters). There is a very good paper published on validating clustering techniques. There are 3 criteria defined in this paper for validating your clusters. You can take a look at that.

Also, the link below gives you some code in R for cluster validation. If you want you can try this too-

http://www.sthda.com/english/articles/29-cluster-validation-essentials/97-cluster-validation-statistics-must-know-methods/

$\endgroup$
  • $\begingroup$ Thank you very much for the great answer. I will go through the links you have provided :) If I get any further problems I will let you know. $\endgroup$ – J Cena Apr 11 '18 at 11:32
1
$\begingroup$

You have to use unsupervised learning.

After that, in order to measure accuracy of your model, you should use cluster quality intrinsic and extrinsic metrics. Compute the similarity of the data in each cluster (intrinsic metric), and the dissimilarity between the data of different clusters (extrinsic metric). A good clustering throws data with great similarity in each cluster and great dissimilarity between clusters.

$\endgroup$
0
$\begingroup$

Applying machine learning without labels is called unsupervised learning. These methods are definitely harder to train and evaluate as you have pointed out than supervised learning. I will warn that 200 features is quite a lot, the more features you have the higher the dimensionality and thus the higher the complexity. Unsupervised learning techniques are not well suited for highly complex data.

In general, unsupervised learning assumes that your data is separated into $k$ separate classes. Each with a distinct distribution. The model you will choose will try to estimate the distribution parameters which describes each of the $k$ classes.

What you can do to measure a fitness for your model? You can try to find an adequate balance between the similarity of instances within clusters and dissimilarity between instances in different clusters. The distance between the points within a cluster (intra-cluster distance) should be minimized. Whereas the distance between instances in different clusters should be maximized.

$\endgroup$
  • $\begingroup$ Thanks a lot for your great answer. Could you please let me know if we can perform the approach you have suggested to measure the fitness in weka GUI? Or else are there any tools/APIs that I can use to achieve this? $\endgroup$ – J Cena Apr 11 '18 at 11:29
  • $\begingroup$ What language are you using? Clustering can be done with sklearn using Python. $\endgroup$ – JahKnows Apr 11 '18 at 11:49
  • $\begingroup$ I am using python as you have suggested :) I will try sklearn. $\endgroup$ – J Cena Apr 11 '18 at 12:21
  • $\begingroup$ Also checkout Scipy $\endgroup$ – Aditya Apr 11 '18 at 15:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.