Class weight ineffective in sklearn

Question

I'm dealing with an imbalanced dataset and as usual it's very easy to obtain a high accuracy, but the recall on the less frequent class is very low. I would like to improve on the false negative of the less frequent class.

Focusing for concreteness on the sklearn Random Forest, one possible strategy is to set a class_weight penalizing the errors on the less frequent class and scoring with a sklearn scoring function as ROC.

grid = GridSearchCV(pipe, cv=number_of_cross_validations, n_jobs=1, param_grid=param_grid , scoring = scorer)
classifier = grid.fit(X_train, y_train)

where pipe is pipeline containing the RandomForestClassifier and param_grid contains different class_weight to try.

My question is probably related to this question, indeed class_weight alone seems to not be enough to lower significantly the false negative. As an extreme example, if I set:

 class_weight = {0: 0.0000001, 1: 0.9999999} 

(where 1 is the class with less instances, with a ratio 1:50), I would expect a final classifier predicting nearly always 1, since every time it makes an error the cost to pay is very high. But that's not what happen, the fitted classifier continue to make a significant number of false negative. Why is it so?

Answer 1

This is most likely because sklearn has chosen your majority class for you and you simply need to weigh your classes the other way around with "0" being the majority class.

Answer 2

If your classes don't have a class overlap problem, you might want to try some re-sampling techniques. you can oversample the class with less records, or look into packages like SMOTE ,ROSE.

Good example for SMOTE: https://www.kaggle.com/qianchao/smote-with-imbalance-data#