Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have a big dataset with nearly 200 features. However, I do not have class labels for these data. I want to divide these data into two classes based on these features. I know, when we do not have class labels we have to use some clustering method. However, since I do not have any labels, I am just wondering how to measure the accuracy of the models.
Please let me know the most suitable approach that I should follow?
I am happy to provide more details about my featureset if needed :)
As you know, clustering is Unsupervised learning algorithm. Since you don't know the number of clusters, it becomes hard to find the best possible separation (number of clusters). There is a very good paper published on validating clustering techniques. There are 3 criteria defined in this paper for validating your clusters. You can take a look at that.
Also, the link below gives you some code in R for cluster validation. If you want you can try this too-
You have to use unsupervised learning.
After that, in order to measure accuracy of your model, you should use cluster quality intrinsic and extrinsic metrics. Compute the similarity of the data in each cluster (intrinsic metric), and the dissimilarity between the data of different clusters (extrinsic metric). A good clustering throws data with great similarity in each cluster and great dissimilarity between clusters.
Applying machine learning without labels is called unsupervised learning. These methods are definitely harder to train and evaluate as you have pointed out than supervised learning. I will warn that 200 features is quite a lot, the more features you have the higher the dimensionality and thus the higher the complexity. Unsupervised learning techniques are not well suited for highly complex data.
In general, unsupervised learning assumes that your data is separated into $k$ separate classes. Each with a distinct distribution. The model you will choose will try to estimate the distribution parameters which describes each of the $k$ classes.
What you can do to measure a fitness for your model? You can try to find an adequate balance between the similarity of instances within clusters and dissimilarity between instances in different clusters. The distance between the points within a cluster (intra-cluster distance) should be minimized. Whereas the distance between instances in different clusters should be maximized.
fitness in weka GUI? Or else are there any tools/APIs that I can use to achieve this?
$\endgroup$