Suppose classifier trained with 5 class, and input query content does not belong to any of the trained class data.
Naive bayes provides and random class as a result here. Which classifier deals best in such scenario?
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
You have trained a model to recognize or discriminate between several particular classes. So when having a new test sample (that according to your knowledge) does not belong to any of this class, the model will fit it to the most similar class.
This is, off course, a very general way of putting it. The behavior and things to consider wether you are using a probabilistic graphical model (like Naive Bayes), linear or non linear classifiers, etc, will vary. However, the principle is the same: the model has learnt the relation among the features of the training data you have used to match particular classes.
More specifically, if you are using a probabilistic approach you can use the probability of belonging to a class and define a threshold. So if a probability higher than certain value (let's say 65%) is not given, then your confidence on the result will be low, and you might say "I cannot say wether it belongs to any of this 5 classes with certainty".
Other non probabilistic methods have interesting approaches for scoring a new example in terms of probabilities. Check this link for SVM (https://stats.stackexchange.com/questions/55072/svm-confidence-according-to-distance-from-hyperline), which is actually the approach used in Python scikit-learn: http://scikit-learn.org/stable/modules/svm.html#scores-and-probabilities
If you use Random Forests you could use the voting of each tree to define a rate of confidence in a similar way (https://stats.stackexchange.com/questions/94845/how-to-estimate-confidence-level-for-svm-or-random-forest)
A similar thing can be done with K-Nearest-Neighbors by analyzing the distance of the K or J (where J < K) nearest neighbors and just trust the result if they are close to your test sample in some ratio (very generally said, this can be done in many ways).
Now, methods based on deep learning are trying to learn features in an unsupervised way, so your problem could solved in a more interesting way. However, it would not be strict classification anymore, nor you would have the amount of data and servers required to even try it (but just saying :) )