I am currently using feature selection approaches like filter, wrapper, embedded etc.
All these methods give different set of features and I rank them based on their frequency of occurrence in other feature selection approach.
Ex: If Age occurs in all 6 feature selection algorithm, then they will have a rank as 6. If gender occurs only in 2 feature selection algorithm, its rank would be 2.
So I arrange them in descending order and choose the features which has occurred in at least 3-4 feature selection algorithm.
But where I am trying to seek your help is
1) Is there any systematic way where I get only a subset of features which returns best output? I thought genetic algorithm for feature selection will return a feature subset which will give high output. But unfortunately it lists all features with their importance. I don't wish to define a threshold myself to select few. Example is currently Xgboost produces an F1-score of 81 with around 27-28 features but when I manually play around, it gives around 81.9 or 82.1 with just 6-7 features. So I would like to have a way to select this 6-7 features automatically/systematically without human intervention
2) Is there any algorithm, like RFE but which does exhaustive search like genetic algorithm and finally returns us the best feature set? Mix of RFE and genetic algorithm is what I need
3) I wish to have an exhaustive search in feature space and finally provide me the best feature set which will provide best f1-SCORE which is my objective.
I am looking for a systematic way/algorithmic way to do this rather than me deciding to pick features which occurs at least in 3 feat selection algorithms
Hope my question is clear. Can you help me with this?
