I have an imbalanced dataset. My classes are 0 and 1. The number of 0 class instances is about 20 times more than the number 1 class instances. I know that I should apply oversampling after train test split (to not having over-optimistic results.) and it is better to have original data in the test set, not synthetic.
But there is something that I don't understand how to deal with it. Let's say:
- I have a data file of 525 lines. 25 lines belong to class 1, 500 lines belong to class 0.
- I apply 5-fold cross-validation.
- At each time, I split 420 lines for train and 105 lines for the test. Then I oversample train data, so my train data becomes balanced (with an equal number of 0 and 1 classes).
- But in my test data (105 lines), I have 5 instances of class 1, 100 instances of class 0. After doing classification using training model and test data, I see the imbalance here and I cannot interpret the results (confusion matrix, accuracy, f1-score, tp, fp etc.).
I need to test more instance of class 1. I am not able to interpret results with a class rate of like 5/100.
Does anyone have any idea how to do it? Thank you.