4
$\begingroup$

As we know, an imbalanced data-set has a disadvantage of training a model for deep learning. However, I don't know how to explain it with mathematics?

$\endgroup$
4
$\begingroup$

Let us say you have a skewed class (which is linearly inseparable, otherwise the prediction is bound to be correct always) like this:

enter image description here

Image Credit: Learning from Imbalanced Classes

Now you can clearly see, if you move the boundary downwards you will be miss-classifying much more blue dots than red dots. And we know the cost function of a ML algorithm roughly tries to reduce the miss-classification (by converting it into a real valued loss optimisation problem, instead of discrete mis-classification reduction).

Think of it as democracy, each individual point tells the classifier what to do. The classifier will always listen to majority points. In this case the classifier will be miss-classifying much more blue points rather than red points if moved downwards. So the bias is set to high, so the line moves upwards.

Mathematically speaking: A classifier will always try to minimise its loss function, which roughly corresponds to making correct classifications (making correct classifications is discrete, optimising loss is not). So, it will position the boundary as such to make minimum number of mistakes disregarding class type. In general if a class has more points in the data-set, a classification boundary will tend to to make more miss-classifications of that class compared to class with less points. Hence optimising the class with higher points becomes more of a priority for th classifier, and it sets its bias accordingly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.