1
$\begingroup$

I'm building a binary text classifier, the ratio between the positives and negatives is 1:100 (100 / 10000).

By using back translation as an augmentation, I was able to get 400 more positives. Then I decided to do up sampling to balance the data. Do I include only the positive data points (100) or should I also include the 400 that I have generated?

I will definitely try both, but I wanted to know if there is any rule of thumb as to what to do in such a case.

Thanks.

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
0
$\begingroup$

Imbalance class is a problem when you have small data in most cases. Your data ratio 100:10000 in this case for your model to do well you should increase the records related to minority class. Now no thumb rule exist in ML (read No Free lunch in ML). Unfortunately you will have to try three scenarios and see what works best for you :

  1. Upsampling by using only actual data
  2. Upsampling by creating new synthetic data by using techniques like SMOTE
  3. Trying a combination of both generate some synthetic data and oversample.
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.