I have an imbalance data set where the imbalance ratio No: Yes is 8:1. If I run classifiers on the groundtruth dataset I got recall and F2 measure for Naive bayes, Logistic regression, random forest. The recall and F2 measure somewhere between 0.500 to 0.200. The AUROC measures varies between 0.860 to 0.799. However, if I apply SMOTE method and make the dataset almost balanced where the No: Yes class ratio 2:1. I got recall and F2 measure better than previous approach and the ratio varies 0.650 t0 0.350 but AUROC getting poor and varies between 0.800 to 0.600.
The dataset is a categorical dataset.
What does this results imply?
If I like to improve the performance of all metrics with compare to groundtruth dataset what should be my approach?