I am doing binary logistic regression on a dataset with very heavy class imbalance. Class 1 is only 1% of data. When I train logistic regressor without class weights I get ROC AUC Score of 0.6269. Which is decent. However, when I see my confusion matrix I see that my model never predicted any 1's at all. So why is my AUC so high? I though AUC is meant to be a good measure for such a case.
Confusion matrix Predicted 0 All True 0 32109 32109 1 1223 1223 All 33332 33332
I know Confusion matrix makes the probability threshold 0.5, so is score saying there is some threshold for which model will give higher recall? How can I get this threshold?
Class precision recall f1-score support 0 0.96 1.00 0.98 32109 1 0.00 0.00 0.00 1223
I only care about precision and recall of class 1.