I am working on a binary classification with 977 rows using different algorithms
I am planning to select important features using wrapper methods.
As you might know, wrapper methods involve use of ML model to find the best subset of features.
Therefore, my question is as below
a) Should I use best hyperparameters even for feature selection using ML model? If yes, why?
b) If no to above question, then am I right to understand that we use best hyperparameters using model building with important features selected above (using a wrapper method)?
c) Is it normal to expect that best hyperparameters from a) and b) should be the same? because both are trying to solve the same objective (best f1-score for example)
Am not sure but feel that doing grid search to find best hyperparameters for feature selection and model building separately seems to be a overkill.
can share your views on this please?