0
$\begingroup$

I am comparing the classification accuracy between Naive Bayes (NBC), SVM and a Neural Network. I am using a Dataset of ~18K and 26 Labels.

In the current state the Neural Network get always an accuracy of >80%, but the NBC and SVM fluctuate between 15% and 80%. They mostly end up near one of the two extrema. The only difference for each run is the splitting of the data in Training/Testing with the model_selection.train_test_split() function of sklearn

For the implementation of the classifiers I am also using the classes and functions of sklearn.

enter image description here enter image description here

I highly suspect the problem in my data but I am already doing the basic preprocessing with stop words, lowercasing, etc.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I recommend you to use the "stratify" attribute of the train_test_split function in order to have a good distribution of the classes (this will avoid the case where there is a support of 0 on the class number 25).

Finally, if you see a big variance depending on the dataset, I think a cross validation is interesting.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Thanks a lot. The stratify parameter made it work. $\endgroup$
    – Adrian
    Commented Jul 19, 2022 at 11:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.