1
$\begingroup$

Suppose classifier trained with 5 class, and input query content does not belong to any of the trained class data.

Naive bayes provides and random class as a result here. Which classifier deals best in such scenario?

$\endgroup$
2
$\begingroup$

You have trained a model to recognize or discriminate between several particular classes. So when having a new test sample (that according to your knowledge) does not belong to any of this class, the model will fit it to the most similar class.

This is, off course, a very general way of putting it. The behavior and things to consider wether you are using a probabilistic graphical model (like Naive Bayes), linear or non linear classifiers, etc, will vary. However, the principle is the same: the model has learnt the relation among the features of the training data you have used to match particular classes.

More specifically, if you are using a probabilistic approach you can use the probability of belonging to a class and define a threshold. So if a probability higher than certain value (let's say 65%) is not given, then your confidence on the result will be low, and you might say "I cannot say wether it belongs to any of this 5 classes with certainty".

Other non probabilistic methods have interesting approaches for scoring a new example in terms of probabilities. Check this link for SVM (https://stats.stackexchange.com/questions/55072/svm-confidence-according-to-distance-from-hyperline), which is actually the approach used in Python scikit-learn: http://scikit-learn.org/stable/modules/svm.html#scores-and-probabilities

If you use Random Forests you could use the voting of each tree to define a rate of confidence in a similar way (https://stats.stackexchange.com/questions/94845/how-to-estimate-confidence-level-for-svm-or-random-forest)

A similar thing can be done with K-Nearest-Neighbors by analyzing the distance of the K or J (where J < K) nearest neighbors and just trust the result if they are close to your test sample in some ratio (very generally said, this can be done in many ways).

Now, methods based on deep learning are trying to learn features in an unsupervised way, so your problem could solved in a more interesting way. However, it would not be strict classification anymore, nor you would have the amount of data and servers required to even try it (but just saying :) )

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.