I have trained a neural network multi-class classification model with around 150 classes having around 85% accuracy. Once the model is trained and deployed, it's predicting on new data and I am saving the logs. Now I have to detect those data-points which is wrongly predicted by the model. For example, The model predicted on 10 data-points, out of which there might be 3 data-points which are wrongly classified by the model. Is there any way to get those data-points? I have the following data.

  • The deployed model.
  • The data-points.
  • Corresponding predicted classes by the model.
  • Confidence scores on the prediction.

I know that using confidence score I might get some idea on wrong prediction, but I am thinking is there any other way to get that?


Sadly, this isn't possible. Simply put, if you could know for sure that your model made a mistake, then, your model will never make mistakes.

This is the painful reality of deploying models in practice. You will need manual validations to confirm whether your model performed correctly. In some systems, you may be able to outsource this to the user itself (i.e. give them an option to say the model was wrong), but you'll likely want to run independent validation.

Now, there are indeed ways to make this part less painful. First, as you said, you should take a look at the confidence scores. It should indeed be less necessary to verify predictions where the model is highly confident. You should also use your confusion matrix to figure out which predictions are more crucial to verify. Indeed, some mistakes may be less harmful than others.

What you could also do is experiment with building auxiliary binary models that take in the input, the model output, and predicts whether the model made a mistake. This isn't guaranteed to work, but it may give you better confidence scores for your model.

  • $\begingroup$ your response seems promising and practical. I would like to understand more on the auxiliary binary models that you have mentioned. Could you please elaborate more on the same. $\endgroup$ – Src Jan 11 at 11:52
  • $\begingroup$ If you use confidence scores to identify potentially wrong labels, you basically build a simple binary classifier. What I suggest is building a more elaborate classifier, which uses the model's outputs but also the input itself to know whether it might be wrong or not. You could train this additional model by taking the data points in your training set that your model still makes mistakes on. $\endgroup$ – Valentin Calomme Jan 11 at 12:48
  • $\begingroup$ Great! I will try that out. thanks a lot! $\endgroup$ – Src Jan 12 at 4:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.