I have seen a recurring theme in real-world problems I've worked with, where the problem looks something like "build an image classifier that recognizes classes A, B, and C but if the input is not those classes, don't recognize it". This features in other questions here from time to time, notably here and here and here; however, I have had difficulty finding a general solution to what seems to be a common problem.
The approaches I've seen so far seem to fall into roughly two categories:
- Treat it as an imbalanced class problem. Usual tricks of dataset balancing, under/oversampling, and class weight all apply here. However, the "other" class tends to be broad and diverse, essentially making the CNN try to encode all the information for "other" in the network. While this works "good enough" for some problems, I have seen it perform poorly and generally it seems like a square peg in a round hole.
- One class classification/outlier detection based approaches. In particular, the use of siamese and triplet networks are in a similar line of thought by trying to separate image feature vectors more clearly. Most of the research I've seen tends to focus on image similarity search or verification rather than applying to a multi-class + negative class problem directly. However, the notion of a contrastative or triplet loss to separate a more tightly definable class from everything else seems like a much more sound approach. Perhaps something like training the feature vector and then using one class SVM or some such?
Is there a general strategy for this situation like finetuning CNN's for normal classification? It seems as though some general solution that works much like finetuning plus a contrastative loss of some sort at the feature vector layer likely exists but I'm just not aware of it.
