1
$\begingroup$

I understand that both training and testing sets should have the same distribution and also understand that we should not touch the test set (in terms of oversampling). But we know that oversampling the training set (specifically in multiclass classification) totally changes the distribution of the training set. For example:

  • The distribution of my training set before oversampling is: 90%, 5%, 3%, 2% [for classes A, B, C, and D]
  • The distribution of my training set after oversampling is: 25%, 25%, 25%, 25% [for classes A, B, C, and D]
  • The distribution of my training set using stratified cross-validation is: 90%, 5%, 3%, 1% [for classes A, B, C, and D] -->as stratifying keeps the distribution of the original data.

Could someone please explain why do we use oversampling when both training and testing sets need to have the same distribution?

Improve this question
$\endgroup$
3
  • $\begingroup$ I'm attaching this link, for your reference. If you still don't understand I can write an answer for you with different example. $\endgroup$
    – Toros91
    Commented Aug 21, 2019 at 4:04
  • $\begingroup$ Thank you. This article suggests following these steps: 1. Inside the CV loop, get a sample out and do not use it for anything related model building. 2. Oversample your minority class, without the sample you already excluded. 3. Use the excluded sample for validation, and the oversample 4. Repeat n times, where n is your number of samples I am doing the exact same thing in my code. In step #1 of this process, we are excluding the validation set and oversampling the test set -->making the distribution of training and testing set completely different. $\endgroup$
    – Sarah
    Commented Aug 21, 2019 at 4:17
  • 2
    $\begingroup$ Yes, this article tells you why and why not with good examples. $\endgroup$
    – Toros91
    Commented Aug 21, 2019 at 4:20

1 Answer 1

Reset to default
1
$\begingroup$

Could someone please explain why do we use oversampling when both training and testing sets need to have the same distribution?

We use it because train and test set does not have to have the same distribution of labels. What is important is that the test set comes from the same distribution as your "real" data so that it provides a reliable measure. Post split the training set can be sampled as you see fit, as long as the results are good on your test set.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.