While calculating feature importance in random forest based on gini impurity(MDI offered by sklearn) or via correlation plots, few features with lesser amount of valid data fails to show it's real importance, let's say we have returns and thus have very less return values across sales. So how do we deal with such situations ?
2 Answers
$\begingroup$
$\endgroup$
If the feature is important such as return, you need missing value treatment. You have two options: Remove records with missing value Or build a unsupervised model to predict return.
If you do not want to treat missing value, then fine tune parameters of random forest . Max features should be 1 . This will ensure return is included. An alternate better solution is to provide sample weight should be such that more weight to non null return records.
$\begingroup$
$\endgroup$
Is it an imbalanced dataset case?
If yes, there are some Random Forest algorithms that could help like the Balanced Random Forest or the Synthetic Minority Oversampling Technique.
https://medium.com/sfu-cspmp/surviving-in-a-random-forest-with-imbalanced-datasets-b98b963d52eb
Bagging could be also interesting:
https://machinelearningmastery.com/bagging-and-random-forest-for-imbalanced-classification/
answered Oct 27, 2022 at 8:31