I am a newbie to machine learning fields. One question I always have when dealing with data problem is that whether feature selection helps the performance of prediction?
I understand that if you have small number of observations but relatively large number of features, it helps to select important feature before prediction. (Right?)
On the other hand, if the sample size is really big (n>>p), does feature selection before prediction help? Especially in random forest, XGBoost, etc, they can "ignore" the features that contributes very less to the model (based on my experience, even I put small number of noise variables, the performance does not change much). Do we need to do a feature selection before these algorithms?
Thanks! Enlighten me!