I am testing different models (SVM, Logistic Regression, Naive Bayes, Random Forest) for predicting the class of a spam email. My target is a binary variable. I am analysing only text, no other fields. My dataset includes
Label 0.0 3333 1.0 768
As you can see there is a big problem with classes imbalanced. I read about the use of downsampling and upsampling, so I applied them before training and testing the dataset. I got good results in terms of F1, recall and accuracy for upsampling (above 88%; max 97%), bad for downsampling (<=76%). For instance:
Down precision recall f1-score support 0.0 0.79 0.43 0.56 102 1.0 0.61 0.87 0.76 114 Confusion Matrix: [[ 49 60] [ 12 100]] Up precision recall f1-score support 0.0 1.00 0.85 0.91 873 1.0 0.87 1.00 0.94 884 Confusion Matrix: [[772 141] [ 20 822]]
I would like to ask you if these values can be considered good results or they can't. I am considering a publication (not only to include similar analysis), so I would like to check if such results can be considered reliable, despite of the imbalance.
Any suggestions and advice will be greatly welcome.