What is the best way to classify data not belonging to set of classes?

Question

I am building a multi-class support vector machine (8 classes to be precise) on an image dataset of pre-defined classes. And then I thought of a question:

What if I have an image that doesn't belong to the set of predefined classes, what would be the outcome?

So I decided to experiment with it and the result was very bad. I got a higher accuracy for images that don't belong to any of the classes. Some images gave 98% accuracy, that they belonged to a particular class, even though my expectation should be that they should should have a very low accuracy.

I also tried using OnceClass SVM to first predict if it's part of the class or not. If yes, then what's the label? (Meaning I have 2 models).

But this doesn't seem to work as the OneClass SVM couldn't classify the "other" images well. Now I am running out of ideas of how to go about it.

How can I approach this problem?

Answer 1

Just a proposal of a method to try out.

Stage $1$: Use one-class SVM to assign those images that do not belong to the set of predefined classes as the $9$-th class.

Stage $2$: For those images that passes through your filter, let the multi-class SVM assign them to one of the $8$ classes.

Answer 2

Since, normally you can not have an overview of what kind of different images other from the 8 predetermined classes you have, it sounds like an outlier detection problem. Maybe you could use some out of the self outlier detection algorithm.

Another approach that comes to mind is to perturbate the image and feed it to the classifier. I guess that if the classifier picks on random noise to determine a class, then this would probably lead it to assign the image to a different class.