# The Merits of Feature Reduction Routines

I am interested in learning what routine others use (if any) for Feature Reduction/Selection.

For example, If my data has several thousand features, I typically try {2,3,4} things right away depending on circumstances.

1. Zero variance/Near zero variance

• Using R package caret, nzv
• I find a v.small percentage is zero variance and a few more are near zero variance.
• Then by using nzv\$PercentUnique I may remove the bottom quartile of features depending on the range of PercentUnique's.
2. Correlation to find multicollinearity

• I find the correlation matrix and remove values > 0.75 and remove.
• I have seen others use correlations > 0.5 or 0.6, but don't have any references for it.
3. Boruta / Random Forest

• Love Boruta package but it takes a while.
• Then here again use Forward Feature Selection.
4. PCA

• Depending on the nature of the data I will try PCA last.
• If the model must be explainable then I skip this.
• I may use several criteria: 80, 90, 95% error explained
• Forward Feature selection, look for first ~3:10 orthogonal features

NOTE: I am not suggesting this is the best/worst routine but I'm opening the floor to civil debate. If you need a definition for Civil Debate see Wikipedia.

• honest question, but stackexchange sites like datasciense.se are not for debating things but for objectively defined questions and answers Aug 22, 2020 at 6:30
• @nikos-m After thinking about your post, I remembered this, datascience.stackexchange.com/questions/34357/… Granted, It is not a code related question like S.E.but on topic. I'm torn... Aug 22, 2020 at 17:58
• This older post discusses the merits of procedures, datascience.stackexchange.com/questions/14864/… The question becomes, 'are people willing to broaden the scope here?' Aug 22, 2020 at 18:13