Is classifier able to say there's no-such-case?

Question

I am a starter in ML and I need some help...

The problem

Assume that I have a classifier which can classify left hand / right hand well. I am curious whether it can decide whether there's a hand in the image?

Because if it is not the case, I have to collect additional data that are labeled as "no-hand". But there's so many things that are "no-hand" such like dogs, cats, flies... I don't think it is possible to collect all those possible "no-hand" data..

My intuition

Would it be reasonable that we use some activation functions like [sigmoid] as the final outputs for the classifier(neural network) and we consider the outputs for both left/right classes as the scores that classifier think how far the input is from the left/right hands classes.

Then if the outputs are below some threshold for both left hand and right hand we can say there's no hand in this image input.

Answer 1

Taking your questions one after the other:

Assume that I have a classifier which can classify left hand / right hand well. I am curious whether it can decide whether there's a hand in the image?

No. It cannot. A model which is trained on hand/no-hand can do that. Not this. At most, it shall predict one of the either (left/hand) with a very low confidence score. But, it definitely is not a nice idea to use it for a hand/no-hand identification.

But there's so many things that are "no-hand" such like dogs, cats, flies... I don't think it is possible to collect all those possible "no-hand" data

You are trying to do build a binary hand.no-hand classification model, not a hand/cat/dog/... style multi-label classification model. So, you just need to have a bunch of images with a hand in it, and some more which doesn't have one.

Would it be reasonable that we use some activation functions like [sigmoid] as the final outputs for the classifier(neural network) and we consider the outputs for both left/right classes as the scores that classifier think how far the input is from the left/right hands classes.

Possible, but not advisable.

Answer 2

Many classifiers do not directly give you L or R. They will give you the option which has a higher decision metric. For Naive Bayes this would be the class with the higher probability between $p(L|x)$ and $p(R|x)$. This is true for most classifiers.

What you can do is set a hyper-parameter (if you have some anomalous instances) to determine how well contained within a class a certain novel example must be in order to be classified as pertaining to that class.

For example, if $p(L|x)=0.01$ and $p(R|x)=0.05$ then we can reject both hypotheses and say the novel example $x$ belongs to neither class.

You should look further into anomaly detection algorithms, you can also use p-value through the GLRT to test if your prediction falls enough within the class such that it can be classified as such. For example, set you p-value, $p = 0.05$, if the p-value of a given example falls below this threshold then do not attribute a class to it and flag it as anomalous.

There's A LOT of different ways this can be done, including using some clustering or NN techniques.

---

In short, you can wrap your classifier with an anomaly detection algorithm to catch hypotheses that are below an acceptable bound.