I have a dataset with 85k positive labels and 53k negative labels. For this use-case, I am trying to maximize my efforts to the negative class (accurately identify true negatives, and minimize false negatives).
Currently, I am able to train a xgboost classifier to 71% accuracy and my confusion matrix looks like this when benched against a test set
[ 3890 | 8887 ] [ 844 | 20044]
Again for this task, I'd really like to improve my recall and minimize false negatives. However, even in the models' current state, if I try to submit 43k new records for prediction (results are unknown), my model predicts that all 43k records are non-compliant.
Given this information, my two questions are as follows:
- How irregular is this occurrence. I thought that skewing to the majority class was possible, but not the minority.
- Are there any 'best practices' that can be applied to reduce the aggressiveness of my model? not only does it predict that all claims are negative (0), but it does so with very strong confidence (minimum 'probability of occurrence' is above 90%). Again, less than 40% of the records in my trainset are of the minority class, so I don't understand why it's skewing this hard.