I've developed an algorithm to define conditional entropy for feature selection in text classification. I'm following the formula at Machine Learning from Text by Charu C. Aggarwal (5.2.2). The author mentions that Conditional Entropy values are between (0, log(number of classes)) in which my case is (0, 0.6931472).
The author also mentions that features with the largest values can be removed, but he don't give further information about the criteria to define 'largest' (is it only the max value of entropy or a set of the largest entropy values?)
Have you ever guys applied Conditional Entropy for feature selection? If so, based on results, what criteria was used to define features to be removed.
Here a summary of my Conditional Entropy results:
E.tj. Min. :0.5701 1st Qu.:0.6562 Median :0.6563 Mean :0.6558 3rd Qu.:0.6564 Max. :0.6564