I have data with 103 columns. I would like to understand which algorithm is best for feature selection and what may be the logic to call any feature as best.
I run below feature selection algorithms and below is the output: 1) Boruta(given 11 variables as important) 2) RFE(given 7 variables as important) 3) Backward Step Selection(5 variables) 4) Both Step Selection(5 variables)
I not able to decide which one to pick up; with domain knowledge it appears I must pick up results from Boruta (as it is giving most number of variables and all seems important).
However I don't find any concrete reason to pickup the best combination.