1
$\begingroup$

We have a dataset with class A as 10% only and Class B as 90% . Let say we did undersampling or oversampling on training data and we made 50% of class A and 50% of class B. But in reality the data distrubution is skewed. Because if we do undersampling or oversampling then our model will give more importance to minority data.

So how does oversampling or undersampling on training data is going to help during the testing on real time data? Because we can not do the undersampling/oversampling over the testing data.

$\endgroup$
3
  • $\begingroup$ Hi @Professor, welcome to the site. If you find the answers to your question useful, please consider upvoting them. Also, please consider accepting one (with the tick mark βœ“ next to it) if you consider it correct or, alternatively, please describe in a comment why you consider it incorrect or not clear enough. $\endgroup$
    – noe
    Commented Nov 30, 2023 at 10:57
  • 1
    $\begingroup$ stats.stackexchange.com/questions/357466/… $\endgroup$
    – Dave
    Commented Nov 30, 2023 at 11:27
  • $\begingroup$ Hi, I really appreciate your help. but the thing is I need minimum 15 reputation in order the upvote the answer. Once I have that, for sure I will upvote it. $\endgroup$
    – XGB
    Commented Nov 30, 2023 at 19:26

1 Answer 1

2
$\begingroup$

The key here is how you define "help" regarding the measurement of performance. Oversampling/undersampling may not help increasing accuracy. However, it may help increase other performance measurements, like the AUC, which give importance to not misclassifying the minority class.

Here you have an illustrative example taken from here:

[...] let's take a look at a binary classification problem where the positive class is dominant. Say, we have a sample distribution and a randomly-predicting model with default accuracy 0.8 (predicts positive constantly without even looking at the data). You can see that this model will return a high accuracy score, although its precision is rather low

$π‘ƒπ‘Ÿπ‘’π‘π‘–π‘ π‘–π‘œπ‘›=\frac{𝑇𝑃}{𝑇𝑃+𝐹𝑃}$

because the number of false positives will grow and therefore the denominator is larger ...

What the AUC on the other hand does, is that it notifies you that you have several wrongly classified positives 𝐹𝑃 despite the fact that you have a high accuracy because of the dominant class, and therefore it would return a low score in this case.

$\endgroup$
2
  • $\begingroup$ Thanks. Can you elaborate more on how the AUC will improve by this? $\endgroup$
    – XGB
    Commented Nov 30, 2023 at 10:09
  • $\begingroup$ I updated my answer with an example. $\endgroup$
    – noe
    Commented Nov 30, 2023 at 10:57

Your Answer

By clicking β€œPost Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.