0
$\begingroup$

In a certain binary classifcation problem I am getting a AUC of 1 and Accuracy,FI,Recall,Precision of ~99.7 both in train,test and holdout sets. But when I run the model on unlabelled data which I want to predict, I feel there is something wrong, as my model is able to predict only 100 1s and the rest 1.3 lakhs tagged 0, which seems very wrong.

What should I make out of this? Is there really a problem with my model?How can I troubleshoot this as its performing well on many sample splits of train data

Improve this question
$\endgroup$
1
  • $\begingroup$ What is the proportion of positive negative instances in the training data and test data? $\endgroup$
    – Erwan
    Aug 14, 2022 at 22:17

1 Answer 1

Reset to default
0
$\begingroup$

Their seems to be class imbalance issue one of your class is Minority while the other is majority. So you will get high accuracy even if the minority class is not having good accurate . one of the way to check this is to find out class wise accuracy .you can do this by using classification report function of sklearn. there are separate notes on class

https://datascience.stackexchange.com/a/113453/138894

Improve this answer
$\endgroup$
3
  • $\begingroup$ I'm not sure about this, I don't see how the AUC can be 1 in this case. $\endgroup$
    – Erwan
    Aug 14, 2022 at 22:16
  • $\begingroup$ yes all the other metrics are around 99p andAUC is 1 . prop of 1 :0 40:60 $\endgroup$
    – Scope
    Aug 15, 2022 at 2:59
  • $\begingroup$ Can you share code ? $\endgroup$
    – amol goel
    Aug 15, 2022 at 3:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.