0
$\begingroup$

Please can you help me with confusion matrix. I've implemented the XGBoostClassifier. After fitting the model when I looked to the confusion matrix to view the performance on Test data. The confusion matrix was showing Positive class as Negative class. My positive class is Submitted and Negative class is Not Submitted.

My dataset is extremely imbalanced. Positive class is very small in size.

Not Submitted:1423948 count

Submitted:5775 count

I've attached the photo of confusion matrix

The reason why I'm asking is because the model which I've created is not performing well. I want to do a hyperparameter tuning. I've used ROC_AUC curve which gave around 99% score. But confusion matrix is showing something else.

enter image description here

Improve this question
$\endgroup$
  • 1
    $\begingroup$ Confusion matrix is based on a cutoff score. What cutoff did you use? Often the default is 0.5. Is that the appropriate cutoff for your problem? Need to balance the benefits of TP and TN vs the costs of FP and FN. Each of these benefits and costs may be different. Optimize for the statistics of the model vs optimize for the usage of the model. Look at the cost/benefits then construct the confusion matrix with the right cutoff. Then try retraining to get a better total cost/benefit. $\endgroup$ – Craig Jan 10 at 15:22
  • $\begingroup$ consider re-sampling the data to fix imbalance if possible $\endgroup$ – Nikos M. Jan 10 at 18:37
  • $\begingroup$ @Craig I was not aware about if there is any cut off in Confusion matrix. Are you mean to say about ROC_AUC curve? where we have TPR vs FPR graph? $\endgroup$ – Pramod yadav Jan 11 at 14:01
  • $\begingroup$ The roc curve sweeps through all cutoff values. There are many references to learn about ROC curve and AUROC. Here is one stats.stackexchange.com/questions/145566/…. The confusion matrix is built from 1 cutoff value. How do you classify a prediction to one of the squares? $\endgroup$ – Craig Jan 12 at 13:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.