3
$\begingroup$

I have an AlexNet architecture that predicts in a supervised-way, whether a given image belongs into classes 1-10. But it should also recognize if it doesn't fit into any of these classes.

My thought was that I just add an 11th class and label all my negative-data with this class and train the softmax-cross-entropy loss with Gradient Descent Optimizer (Adam) as usual.

Now my question is if this is a valid approach on solving my problem?

I have around 7.000 images per class and could obtain up to 70.000 negative examples. I have the strong feeling that training with 70.000 negative-examples my network would always predict the outcome to belong to class 11 (not matching any other). Am I correct with the assumption? If yes: How many negative examples would do the job?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Your approach of adding negative examples to your data set, replacing the 10-output softmax final layer with an 11-output softmax, and retraining is certainly valid.

You need these additional examples because the network can only learn to classify categories that it has seen before.

Retraining the final layer only is a good first attempt because it is the cheapest, but if you don't get good results you can try fine tuning one or more additional layers at the end of the network.

All these techniques fall under the topic of Transfer Learning.

Regarding the second part of your question about how many negative examples to include - this is known as Class Imbalance Problem in Machine Learning.

Whether your model will always predict the negative class if you include many negative examples depends on the data and the model, but it is certainly a possibility.

The safest approach would be to undersample your negative examples and include only 7000 to match the frequency of other classes in the training set. This should be a good first try, but if it doesn't work you should consider a couple of questions:

  1. What is the expected distribution of my target population?
  2. What is the cost of misclassifying each type of example?

And then make an informed decision about how to proceed, which may involve either sampling negative examples differently, or adjusting your performance metric to something like AUC (which can guide early-stopping, for example).

Additional material: A systematic study of the class imbalance problem in convolutional neural networks

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for the really quick answer! I will look into your suggestions. Bless you, Jonathan $\endgroup$
    – Jonathan
    Feb 11, 2018 at 22:37
  • $\begingroup$ I would suggest against re-balancing the classes. I am of the philosophy that more data is better. Instead, you could change the probability threshold that identifies the image as the "non-object class." Then, as Imran suggests, test the threshold setting using AUC. $\endgroup$
    – user18764
    May 13, 2018 at 12:13
  • $\begingroup$ I'd have thought having an extra "none of the above" class is superfluous - your negative samples simply have the label "not this" for all classes and should fit your training regime directly. $\endgroup$
    – drevicko
    Jun 14, 2019 at 5:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.