2
$\begingroup$

Caveat: I'm aware that imbalanced data questions are a dead horse, but I haven't found an answer to this flavor of it directly.

When working with highly imbalanced data (e.g. binary class cases), the common wisdom is to try training on an upsampling of the minority class, downsampling of the majority class, some combination of the two, or something like SMOTE. The potential gains of doing so are obvious if one accepts default model thresholds and tunes for matrix-based metrics like accuracy or f1. However...

It is my experience that if one instead optimizes for a threshold-invariant metric first (like ROC AUC or PR AUC or similar) and then adjusts the threshold to meet the specific business need, resampling loses its benefit swiftly and often hurts more than helps.

Is the common wisdom of balancing class data (or its various flavors) then folklore given the above strategy? If not, in what cases should one generally expect it to help, and in what cases hurt?

$\endgroup$
2
  • $\begingroup$ see discussion on this similar post datascience.stackexchange.com/questions/90964/… $\endgroup$
    – Nikos M.
    May 19 at 19:06
  • 1
    $\begingroup$ the short answer is yes, artificial sampling is not needed in many cases and threshold adjustments (among other approaches) can indeed help $\endgroup$
    – Nikos M.
    May 19 at 19:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.