# Model that predicts probability of correctness of another model

Problem:

Given a neural network for image classification with $$1000$$ classes, the objective is to create another model which will output the probability of the neural network giving the correct prediction for a specific input image.

Thoughts:

My ideas so far have been:

• Creating a convolutional network and training it with raw images together with a label which indicates if the first NN predicted it correct or not.
• Creating a fully connected network and training it with outputs of hidden layers/features of the first NN instead of the raw image, together with the labels as before.
• Creating a fully connected network and training it with the top-k outputs of the softmax layer of the first NN together with the labels.

The first method yielded an accuracy of $$0.51$$, the second $$0.58$$ and the last one $$0.79$$.

Can you suggest another method (or a modification of one from the above) which can achieve greater accuracy?

• Very insteresting. To see this with an example, the model for the original task has $0.69$ accuracy on previously unseen images. Now, the QE model, when trained with the hidden layer ouput of the original has $0.58$ accuracy and $0.79$ when trained with the softmax ouput of the original. Is there a way to evaluate the QE model's performance with reference to the original model's performance? In other words, how accurate can the best QE model be for an original model of $0.69$ accuracy? May 15 at 13:40