I am currently dealing with a binary classification task on imbalanced data with the following distribution:

y_train: 4981 positive / 863894 negative samples  
y_test:  128  positive / 128309 negative samples 

The goal is to aim for a high precision (as little false negatives as possible).

How do I go on about choosing the weights for the random forest?

I tried to balance out the y_train ratio by assigning weight 1 to "negative" and 173 to "positive", but that still caused all the samples to be assigned to negative.

At this point, should I already consider this a problem with the features used, or should I try to assign higher weights with GridSearch (roc_auc as scoring parameter) and set the decision-threshold higher first?

  • $\begingroup$ As an idea: you could use boosting since it often works well on imbalanced data and there are tools to specify the class weights, e.g. in LightGBM (pos_bagging_fraction) or in Catboost (scale_pos_weight). $\endgroup$
    – Peter
    Aug 8 '19 at 11:49
  • $\begingroup$ @Peter Thanks for the suggestion! I am planning to try gradient boosting as well, but for my first attempts I will go with random forests as they train faster and have a class_weight option as well $\endgroup$ Aug 8 '19 at 13:02
  • $\begingroup$ When you use L1 regularization (aka lasso) you can achieve „automatic“ best feature selection. This would also be an argument for boosting, I guess $\endgroup$
    – Peter
    Aug 8 '19 at 13:53

I totally missed out on the fact, that the RandomForestClassifier accepts the values “balanced” and “balanced_subsample” to automatically assign the weights as per documentation


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.