How to classify images Neural Network didn't trained to Understand

Question

Let's say I trained a Convolution neural network to Identify Cats , Dogs and wolves . But suddenly I feed it pictures of rabbits and Lions. so how can I classify those as pictures as "Other"

I tried to do this by Adding "Sigmoid" activation functions and getting probabilities for each selection. I thought probabilities for each selection might be different. I thought If I feed a Rabbit picture to NN it will out 20% cat ,44% dog and 34% wolf. That way I can clearly figure NN is confused. But Unfortunately it gives me results like 70% cat , 10% Dog , 20% wolf.

Can You suggest me a Way to fix this problem ?

Answer 1

Unfortunately, a neural network is only able to compute probabilities on labels that it has been trained to recognize. In your model, you only have three identified labels and presumably trained on a data set that only includes the three classed. So your model is evaluating everything in terms of those three labels. If you feed in images of a car, it is going to give you the probability of the car being a cat, dog, or wolf and the probabilities will add up to 100%.

There are several approaches to try to deal with this problem.

Increase Training Examples

As Nga Dao suggested, add another class others and add a bunch of images that are not part of the target classes with the label others. This is probably the easiest option but it may not produce much better results.

One-vs-Rest Modeling

Create a binary classifier for each class and take the class with the highest probability over a threshold. For example, when classifying a rabbit using the cat binary-classifier, presumably, the probability of a cat will be lower than not a cat. If all probabilities are below a threshold you feel is significant, then label the image as other. If you have two significant probabilities, which might happen when classifying a dog and you have dog and wolf as classes for example, take the class with the highest probability.

You may need to optimize the threshold to get the best performance out of your classifier.

Answer 2

You need to modify the dataset by adding 1 label called 'others'. Now, there are 4 output labels: cat, dog, wolf, and others. Then, you train the model again.

Answer 3

You can try to compute score of matching and assign it to recognized objects. Then you could add additional label: others which will be assigned to objects which score was significantly below threshold. Of course you will need to define lower limit not to assign everything as others.

Or you can only add new label for each new object type, modify targets in your training set and train your model once again.