0
$\begingroup$

I'm been experimenting with tree based classifiers for multi-label document classification. All the trees I've created, however, tend to predict only one or two labels per document. Whereas the training set has about 4 labels per document on average. Furthermore, in my particular application, false positives are much less costly to the business than false negatives. So, if anything, I'd like the tree to be making about 6 or 7 predictions per document.

I'm not entirely sure which parameters control this. I've tried experimenting with tree size but without effect. I'd ideally like to just set a threshold for when a prediction is included, and lower this.

I'm using sklearn (and playing with skmultilearn). Here's an example of a forest configuration:

 from sklearn.ensemble import RandomForestClassifier 

    clf = RandomForestClassifier(
        n_estimators=20,
        criterion='gini',
        max_features=0.5,
        max_depth=68,
        min_samples_split=4,
        min_samples_leaf = 2,
        n_jobs=3
        )
Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
0
$\begingroup$

You seem to experience a class imbalance situation where some classes dominate the others by the number of samples they have, so that your algorithm finds it wise to predict less of the rare classes or non-predict them to decrease the loss at the end. You can manually set the class weights for the Random Forest Classifier, making loss function treat unevenly to different classes, but even in total.

For details you can refer to: https://stackoverflow.com/questions/20082674/unbalanced-classification-using-randomforestclassifier-in-sklearn

Note: Random Forest is not robust to class imbalance, this is a known situation. You can refer to: https://stats.stackexchange.com/questions/242833/is-random-forest-a-good-option-for-unbalanced-data-classification

Hope if I could help. If not, I will be around for further discussion.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for your answer. I do have quite a strong class imbalance. I've been tackling this by generating synthetic documents. But perhaps not aggressively enough. I will check to validate that indeed the lower-represented classes are not being picked, and if that's the case, try a more aggressive strategy or use weights instead as you suggest. Much appreciated. $\endgroup$
    – Neil
    Nov 30, 2018 at 11:10
  • $\begingroup$ You're welcome. You can also do this: Use LightGBM (a state-of-art tree-based algorithm by Microsoft with similar parameters), use 'multiclassova' as classification task and feed is_unbalance = True as a parameter. LightGBM will understand the class imbalance situation and fix it by itself in the framework. $\endgroup$
    – Ugur MULUK
    Nov 30, 2018 at 11:44
  • 1
    $\begingroup$ Hamming loss down from 45 to 20. So yea I underestimated how significant an impact the class imbalance was having. Thanks! $\endgroup$
    – Neil
    Dec 1, 2018 at 10:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.