3
$\begingroup$

I've seen a few posts and papers floating around the web (mostly those related to over/undersampling, SMOTE, and cost-sensitive training) that, when discussing class imbalance, specify that certain algorithms are negatively impacted by class imbalance.

Which algorithms are those? Which are not? How can we figure out whether or not an algorithm or approach will be negatively affected by class imbalance?

$\endgroup$
  • $\begingroup$ I like this question; almost everything I've seen implies that imbalance is bad, almost as a sort of folklore, without clear references. Now, there is the clear problem that a poor model can have high accuracy in an imbalanced dataset (by just always predicting the majority class), but that speaks more against accuracy as a score IMO. I hope there's an answer that is somewhat theoretical and concrete, and not just the obvious answer to your third question: "try it with and without imbalance correction." $\endgroup$ – Ben Reiniger Jul 4 at 1:35
-1
$\begingroup$

[This answer is based on my limited knowledge, please don't hesitate to edit or propose improvements in the comments]

Actually I think it's a bit misleading to say that algorithms can be affected by class imbalance, because it's not exactly the algorithm which is affected it's the evaluation method (I mean "evaluation" in a broad sense including the loss function used by the algorithm during training). Some algorithms may be closely related to a particular loss function or internal optimization strategy, so by association such algorithms have the same weaknesses.

A simple way to see that class imbalance issues completely depend on the evaluation method is to compare micro-average performance with macro-average over the classes in a case where 99% of the instances belong to the same class:

  • micro-average gives the same weight to every instance, so a model which assigns the majority class will look as if it performs very well.
  • macro-average gives the same weight to every class, so assigning the majority class won't work better than random.

So technically the problem of class imbalance could (should?) be seen as a design choice between maximizing the number of instances correctly classified (default evaluation) and any other alternative, for example giving equal weight to every class. But of course it's not practical nor common to design a specific evaluation measure or loss function specific to every problem.

$\endgroup$
  • 1
    $\begingroup$ if you downvote it would be nice to explain what is wrong with my answer... especially when it starts with "This answer is based on my limited knowledge, please don't hesitate to edit or propose improvements in the comments" $\endgroup$ – Erwan Jul 6 at 0:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.