I am doing text classification in python with 3 alghoritms: kNN, Naive Bayes and SVM. I have 3 classes - easy, medium and hard. The accuracy is quite fine. Is there a way to check for new text its exact value? After label encoding 0 is easy, 1 is medium and 2 is hard. So base on new text for example it is classified as medium but I want to know how close it was to easy/hard. Some of my code snippets:

Train_X, Test_X, Train_Y, Test_Y = model_selection.train_test_split(
    df['tokens'], df['class'], test_size=0.3, random_state=42)

Encoder = LabelEncoder()
Train_Y = Encoder.fit_transform(Train_Y)
Test_Y = Encoder.fit_transform(Test_Y)

Tfidf_vect = TfidfVectorizer(max_features=35)
Tfidf_vect.fit([' '.join(arr) for arr in df['tokens']])
Train_X_Tfidf = Tfidf_vect.transform([' '.join(arr) for arr in Train_X])
Test_X_Tfidf = Tfidf_vect.transform([' '.join(arr) for arr in Test_X])
Naive = naive_bayes.MultinomialNB()
Naive.fit(Train_X_Tfidf, Train_Y)
predictions_NB = Naive.predict(Test_X_Tfidf)
print(round(accuracy_score(predictions_NB, Test_Y)*100, 2))

Now when I use Naive.predict() I get 0, 1 or 2. Is there a way to get the EXACT value for example 0,5897237489 which is 1 but I see that it is closer to 0 than 2

If it is not possible with NB, I used SVM and kNN as well:

SVM = svm.SVC(C=1.0, kernel='linear', degree=3, gamma='auto')
SVM.fit(Train_X_Tfidf, Train_Y)
predictions_SVM = SVM.predict(Test_X_Tfidf)
print(round(accuracy_score(predictions_SVM, Test_Y)*100, 2))

KNN = KNeighborsClassifier(n_neighbors=3)
KNN.fit(Train_X_Tfidf, Train_Y)
predictions_KNN = KNN.predict(Test_X_Tfidf)
print(round(accuracy_score(predictions_KNN, Test_Y)*100, 2))

The problem is that Naive Bayes is a classification algorithm that outputs the probability of an input belonging to a class. So you cannot use Naive Bayes to achieve what you want. To get 'in-between' values you need to use a regression algorithm instead. I'm not sure what library you're using, but try to use regression versions Knn and SVM models instead.

  • $\begingroup$ I used both kNN and SVM. How to do it with these algoritms? See updated question. $\endgroup$ – jake-ferguson May 1 '20 at 11:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.