I've read in sklearn's documentation that we have to take special care in balancing the input for a decision tree, but it doesn't tell you what function to use. However, I've found the parameter class_weight. If I use class_weight: balanced as a parameter, will that mean that I can omit balancing the input by hand?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
Yes you don't need to balance your train data by hand. But your test data can still be (truly) unbalanced.
-
2$\begingroup$ But the test data can be unbalanced, right? That wouldn't make the algorithm perform less accurately, if the tree is accounting for the input's unbalances. $\endgroup$– lte__Jan 17, 2018 at 14:34
-
$\begingroup$
$\endgroup$
2
For classification problems, not just decision trees, it isn't uncommon for unbalanced classes to give overly optimistic accuracy scores. There's a few common ways to handle this.
Resamble your data. You can oversample the minority class or undersample the majority class. The end goal is to balance out the data more or less.
Give your model a prior to help inform frequency.
Similar you can pass a weight argument. The weight will penalize the classification function for misclassification of a rare positive cases.
Lastly you can modify the accuracy measurement. A common measurement would be to use F1 statistic instead of just accuracy. Maximize the F1 statistic by cross validation and see if it's stable during testing.
I also advise you to combine these techniques. For example, give your model a prior and use the F1 score. I think you'll find good results going this route.
-
$\begingroup$ can
class_weight="balanced"in sklearn used with cross validation? Looking forward to hearing from you. Thank you :) $\endgroup$– EmJMar 30, 2019 at 21:06 -
$\begingroup$ yes, you pass
class_weightto the model function, and then pass the model tocross_val_scoreor similar $\endgroup$– PaulAug 23, 2019 at 0:14