TL;DR
You can train your classifier as usual and threshold on the prediction probability.
Scikit-learn
You won't get that out of the box, but you can use this simple class to do just that!
from sklearn.tree import DecisionTreeClassifier
from numpy import argmax, max
class MyClassifier(DecisionTreeClassifier):
unknown_class = 'unknown'
def __init__(self, no_class_threshold=0.75, **kwargs):
self.no_class_threshold = no_class_threshold
super().__init__(**kwargs)
def predict(self, X):
preds = self.predict_proba(X)
y_pred = [self.classes_[i] if v > self.no_class_threshold else self.unknown_class
for i, v in zip(argmax(preds, axis=1), max(preds, axis=1))]
return y_pred
Just use this class as you were using the DecisionTreeClassifier before. You can pass all the parameters from DecisionTreeClassifier plus an extra one: no_class_threshold.
The no_class_threshold works in the following way:
IF
prediction_probability > no_class_threshold
THEN
output predicted class
ELSE
output 'unknown'
Changing classifier
You might not want to stay with the DecisionsTreeClassfier, and instead, you might want to experiment with other classifiers. You can do this by inheriting from another Scikit-learn class. Simple as that!
For example:
from sklearn.svm import SVC
class MyClassifier(SVC):
[...]
Choosing the no_class_threshold
You can:
- Choose it manually, to a degree that you feel confident
- Use hyper-parameter tuning to learn it, if you have data points labelled as 'unknown'.
Example
Here is a script you can run to test all the above.
from numpy import argmax, max
from sklearn.svm import SVC
class MyClassifier(SVC):
unknown_class = 'unknown'
def __init__(self, no_class_threshold=0.75, **kwargs):
self.no_class_threshold = no_class_threshold
super().__init__(**kwargs)
def predict(self, X):
preds = self.predict_proba(X)
y_pred = [self.classes_[i] if v > self.no_class_threshold else self.unknown_class
for i, v in zip(argmax(preds, axis=1), max(preds, axis=1))]
return y_pred
if __name__ == '__main__':
X = [[1, 1, 1],
[1, 0, 1],
[0, 0, 0],
[0, 1, 0],
[0, 1, 1]]
y = ['class1', 'class1', 'class0', 'class0', 'class0']
clf = MyClassifier(no_class_threshold=0.55, probability=True, C=1)
clf.fit(X, y)
X2 = [[1, 1, 0],
[0, 1, 1],
[0, 0, 1]]
pred = clf.predict(X2)
print(pred)
