Problem:
Given a neural network for image classification, the objective is to develop an algorithm which decides which images are 'problematic' and the model is probably going to classify them incorrectly.
Discussion:
So far, I've thought of two possible approaches:
- Feed the given image to the model and then analyse its softmax ouput with various metrics (difference between first and second class confidence, entropy, gini index etc).
- Perform some kind of image processing (feature extraction) on the given image, to obtain some features that indicate whether the image is not going to be correctly classified.
Questions:
Can you provide me with more suggestions about the second approach? What type of feature extraction would you think will help distinguish those images?
Any other ideas that are not mentioned here are welcome.