I am working on a text classification use case. The training data has two classes, so the XBBoostClassifier and onevsrest model is classifying the test data into either of the two classes. But my requirement is to classify either into given the classes or if no match found, then set it as 'undetermined" so that i can manually classify the data.
I tried the following OneVsRest Classifier
pl = Pipeline([ ('vec', CountVectorizer(token_pattern = tks)), ('clf', OneVsRestClassifier(LogisticRegression())) ]) pl.fit(x,y) predictions = pl.predict_proba(test.comment_text)
But the sum of the probabilities is one and moreover the probability is above 90 for the class to which the data belongs to.
Please clarify the following points 1. Why the probability is always one? whether it means that the data is mutually exclusive? 2. The probabilities are like this
CLASS 1 :CLASS 2 0.892993358265023 : 0.106808845640795 0.999999742528922 : 2.57685096542208E-07
Does that mean that in first case, probability is only 90% for class 1 and hence the classifier is not able to classify the data properly. However in other case, there is clear cut difference as the probability is around 99%
Can I set the threshold, like 90%, and conclude if the probability is less than 90% , let the users manually classify the data?
Please provide your inputs