Convolutional neural network giving high confidence on wrong classification

Question

I am doing a project on sign language recognition through CNN.

My Dataset

300 images for each alphabet and one special symbol for space. Images are taken from three individuals. Each image has hand centered in white background. Resolution is 640*480. So in total 27 labels and 8100 images.

Samples Images

Code

I am using pretrained AlexNet in PyTorch and have changed fully connected layers according the need of the project. I am training for 10 epochs. Batch size is 32. Loss function is Negative Log Likelihood Loss and optimizer is Adam.

AlexNet(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2))
    (1): ReLU(inplace=True)
    (2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
    (3): Conv2d(64, 192, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
    (4): ReLU(inplace=True)
    (5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
    (6): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (7): ReLU(inplace=True)
    (8): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (9): ReLU(inplace=True)
    (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(6, 6))
  (classifier): Sequential(
    (0): Linear(in_features=9216, out_features=1024, bias=True)
    (1): ReLU()
    (2): Dropout(p=0.5, inplace=False)
    (3): Linear(in_features=1024, out_features=512, bias=True)
    (4): ReLU()
    (5): Dropout(p=0.5, inplace=False)
    (6): Linear(in_features=512, out_features=27, bias=True)
    (7): LogSoftmax()
  )
)

Problem

The problem I am facing is that the model is giving very high confidence about 90-100 % on images that have no hand gesture in them or contain wrong hand gesture. I have tried adding dropout layers, training for more epochs but nothing seems to solve this problem. Is there something wrong with the model or the dataset needs to be changed/updated?

Answer 1

Since your model seems to be doing fine on the correct hand signs, I don't think there is anything wrong. It's a common misconception that you can interpret the model output as a confidence measure. Consider the following example.

Let's say you have a binary classifer and you learn to distinguish numbers between -1 and 0 in one class labelled as 0 and numbers between 0 and 1 in the other labelled as 1. Your training data is in these intervals and is labeled correctly. Your classifier in the end will be something like

$$ y = \sigma(wx)$$

with $\sigma$ as the sigmoid function. When you train, you will quickly learn that $w > 0$. The more you train, the larger $|w|$ will become untill you get something close to 0 for all $x \in [-1,0)$ and something close to 1 for $x \in (0,1]$. All perfect and normal so far.

Now you pass the number $x=10$ to the model. The query does not make sense, as this $x$ does not belong to any of the classes the model was trained on and has nothing to do with the training data. There is no reason to assume that the model would give any useful output for $x=10$. But still, the model will the super confident that $10 \in (0,1]$ since for $x=10$ the model will output a value closer to 1 than for any of the training examples. This is clearly not correct.

I hope this clarifies that the interpretation of the classifier output as confidence can be very misleading. If the classes are close together in feature space and you query with an example that is somewhat between the training data, then you could take it as such, but for input that is not represented in the training data, the numbers you will get don't have any meaning at all.

In your application you could introduce the counterexamples to the training data and label them as "other", but that would be risky as well. If you really need a confidence measure, look into Bayesian Neural Networks (e.g. Bayes-By-Backprop, Dropout or Batch norm), other Bayesian methods or bootstrapping.