# Extracting useful features for k-means clustering

So say suppose I have a data-set with features being either present or not i.e. 0or 1. Now I want to identify the features which really help in the clustering.

Like say I have 4 training examples. Now say I have a feature which is present i.e = 1 in all the training example, thus I can remove the feature as it dies not help me. Now let's talk about 2 more features, if the number of training examples they are present in common is high, they also do not help much in clustering (think of 2 highly overlapping circles in a Venn Diagram). So in this way I want to find features which has a significant impact on the clustering i.e mostly non-overlapping features.

Is there any good way to do this? (my features are all represented in binary, either it is there or it is not).

There are many ways for features selection:

One possible way (Wrapper methods), is to start by clustering with all related features and save the clusters, then cluster again after removing 1 feature and compare the obtained clusters (without the feature) to the clusters (with the feature). If the difference between the results are negligible then you can eliminate that feature.

Filter methods include what you mentioned in your question, where you remove the feature that changes rarely or doesn't change at all which can be expressed mathematically by "entropy", "information gain", "Chi square test"...etc. these concepts measure the impurity of a feature.

To sum up, "Wrapper" methods usually provide better results but they are computationally more expensive.