I created a convolutional network to recognise certain substrings.
For example, the following phrases would be mapped to the "What" class:
What are you doing? What you doing? Whatcha doing?
The following to the "How":
How are you doing? How you doing?
Now, say I have millions of examples and thousands of classes, my network learns relatively well. However, sometimes a new "phrase" that belongs to a never seen class appears. Of course, the model cannot map it to something it has never seen, however, when I look at the output layer values, I was hoping to see all neurons to have small values. My idea was that, I would use the softmax and then if the max value is less than 0.95 or some high value, I would assume the network is unsure, therefore remove that result for manual observation.
However, in many cases, I have seen my network to be almost sure (a neuron with value close to 1) even when it has never seen that class.
For example, say I have the two classes What and How, I would expect:
Whatcha doing? --> "What"= 0.95, "How"= 0.05 How you doing? --> "What" = 0.02, "How"= 0.98 Where you going?--> "What"= 0.6, "How"=0.4
In the last case, the network is not too sure on either, and I would check the input. Unfortunately, this is not working, and I am getting high values (close to 1) neurons for some new input.
How can I fix this? Please, bear in mind, I am happy to sacrifice some accuracy in order to ensure the network never tells me it is sure about new inputs, in other words, I am happy to go and manually check more inputs (even if seen before) rather that let even one unseen input go unnoticed.