I have seen a recurring theme in real-world problems I've worked with, where the problem looks something like "build an image classifier that recognizes classes A, B, and C but if the input is not those classes, don't recognize it". This features in other questions here from time to time, notably here and here and here; however, I have had difficulty finding a general solution to what seems to be a common problem.

The approaches I've seen so far seem to fall into roughly two categories:

  1. Treat it as an imbalanced class problem. Usual tricks of dataset balancing, under/oversampling, and class weight all apply here. However, the "other" class tends to be broad and diverse, essentially making the CNN try to encode all the information for "other" in the network. While this works "good enough" for some problems, I have seen it perform poorly and generally it seems like a square peg in a round hole.
  2. One class classification/outlier detection based approaches. In particular, the use of siamese and triplet networks are in a similar line of thought by trying to separate image feature vectors more clearly. Most of the research I've seen tends to focus on image similarity search or verification rather than applying to a multi-class + negative class problem directly. However, the notion of a contrastative or triplet loss to separate a more tightly definable class from everything else seems like a much more sound approach. Perhaps something like training the feature vector and then using one class SVM or some such?

Is there a general strategy for this situation like finetuning CNN's for normal classification? It seems as though some general solution that works much like finetuning plus a contrastative loss of some sort at the feature vector layer likely exists but I'm just not aware of it.

  • $\begingroup$ One trick I've tried since originally posting this is to take a standard CNN like say MobileNetV2 and add a second loss objective of triplet_semihard_loss hanging off the "feature" layer of the convolutional output. While it still doesn't solve the "other" class fully it can introduce distancing of the feature vectors which seems to help somewhat. $\endgroup$
    – J Trana
    Commented Jan 17, 2020 at 5:47

1 Answer 1


There is no general strategy to anything in machine learning. Moreover, research is scattered over multiple domains so it becomes harder to get your head around, "the best strategy for a case". You can solve a single problem in multiple ways. Coming back to the multi class plus a negative class classification issue, this is one of "many approaches" you can follow:

Hierarchical classification: This is still an open problem. As the name suggests you perform classification on multiple levels starting from a root to internal node to leaf nodes (similar to tree or DAG structure). For eg. Given a digits data set ( 0,1,2,3...9) mixed with some letters (a,b,c....z). On a higher level you need to first classify if its a digit or not i.e. meta-class and if its a digit then you perform lower level classification task to identify the right digit.

There are different variants depending on your knowledge of labels/ groups at different level of hierarchy. For eg. if you don't have a pre-defined meta-class then you might have to perform clustering of similar classes at the higher level before final classification.

Also, you may want to tune the weights in your loss function to identify a particular class efficiently at any level of the classification.

P.S. My knowledge is limited in neural nets context. Definitely there would be more strategies specific to them.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.