1
$\begingroup$

Problem:

Given a neural network for image classification with $1000$ classes, the objective is to create another model which will output the probability of the neural network giving the correct prediction for a specific input image.

Thoughts:

My ideas so far have been:

  • Creating a convolutional network and training it with raw images together with a label which indicates if the first NN predicted it correct or not.
  • Creating a fully connected network and training it with outputs of hidden layers/features of the first NN instead of the raw image, together with the labels as before.
  • Creating a fully connected network and training it with the top-k outputs of the softmax layer of the first NN together with the labels.

The first method yielded an accuracy of $0.51$, the second $0.58$ and the last one $0.79$.

Can you suggest another method (or a modification of one from the above) which can achieve greater accuracy?

$\endgroup$
1
$\begingroup$

This is a problem known as quality estimation (QE) in the application of Machine Translation (there's a regular Shared Task about it). The goal is to train a model to predict the quality of the output of a ML system.

There is an important theoretical obstacle to this kind of task: if such a QE model was able to predict perfectly, then in theory it would be possible to create a perfect model for the original task by trying many different models/parameters until the QE system predicts a high level of quality. What this means practically is that a QE model can only be as good as the system it's trying to estimate since it relies mostly on the same information, otherwise the system is clearly sub-optimal. Essentially the best the QE system can do is to determine how hard it is to correctly predict a particular input image, but it cannot predict with any certainty whether the actual prediction is correct or not.

$\endgroup$
3
  • $\begingroup$ Very insteresting. To see this with an example, the model for the original task has $0.69$ accuracy on previously unseen images. Now, the QE model, when trained with the hidden layer ouput of the original has $0.58$ accuracy and $0.79$ when trained with the softmax ouput of the original. Is there a way to evaluate the QE model's performance with reference to the original model's performance? In other words, how accurate can the best QE model be for an original model of $0.69$ accuracy? $\endgroup$
    – Paris
    May 15 at 13:40
  • $\begingroup$ @Paris in principle the QE model should be as independent of the original model as possible. The way to evaluate the QE model would be to measure if it correctly predicts a low confidence when the prediction of the first model is wrong and a high confidence when the prediction is right (btw I forgot to mention that the task is also called "confidence estimation"). Of course this has to be averaged over a quite large sample of test instances, because it can happen that the original model predicts correctly by chance for example. $\endgroup$
    – Erwan
    May 15 at 19:58
  • $\begingroup$ I'm not really up to date on the topic of evaluation of QE, but these are really interesting questions about what does it mean to predict confidence correctly and how to measure this in a statistically accurate way. $\endgroup$
    – Erwan
    May 15 at 20:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.