Is there a practical threshold to determine whether a dataset is imbalanced or not? i.e. we should packages like imbalanced-learn to do some kind of adjustment like label weight or data generation
$\begingroup$
$\endgroup$
2
-
3$\begingroup$ Note that "unbalanced" data is often not a problem: Are unbalanced datasets problematic, and (how) does oversampling (purport to) help? $\endgroup$– Stephan KolassaJul 6 at 8:41
-
1$\begingroup$ @StephanKolassa brings up a key point: don’t become so preoccupied with wondering if you could do something about imbalance that you forget to consider if you should. $\endgroup$– DaveJul 7 at 10:39
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
There isn't a universally agreed upon threshold for when a dataset is considered imbalanced. However, a common rule of thumb is that if the minority class in your dataset constitutes less than 10-20% of your total data, it can be considered imbalanced.
For example, if you have a binary classification problem with 1000 instances, and only 100 of them are of the positive class (and the remaining 900 are of the negative class), then your dataset is imbalanced.
However, the decision to use techniques like resampling, SMOTE, or class weight adjustment doesn't solely depend on this threshold. It also depends on the performance of your model. If your model is performing well on the minority class despite the imbalance, you might not need to do anything. But if your model is struggling to correctly classify the minority class, then you might want to consider using these techniques.
It's also important to note that these techniques are not without their drawbacks. For example, oversampling can lead to overfitting, while undersampling can lead to loss of information. Therefore, it's important to carefully validate your model and consider the trade-offs.
-
$\begingroup$ If there are 5 classes, then each class having 20% of the total data is very very balanced :-) If you rephrase it as the biggest class is 80%+ of the date, then it feels useful as a rule of thumb; "some classes are <20% of the size of the largest class" feels less useful. $\endgroup$ Jul 10 at 18:28
-
$\begingroup$ True. But there is no rule of thumb though . $\endgroup$ Jul 10 at 19:51
$\begingroup$
$\endgroup$
You can observe the proportions of the classes. Class imbalance might not be a problem when a sample (your data) fits the whole population (the real-world case).
For instance, classifying whether someone would get a heart disease from regular hospital checkups will result in a very small proportion of people with heart disease. Heart diseases in the US show that there are 6.9% of visits that reports heart diseases.
In another case, your data might belong to a more specific subset where people go for checkups when they notice some pain and go for a more specific checkup. Among these people, there would be observable factors that could already lead to the answer. This also causes the features in the data to reveal the answer more easily. In this case, when patients tell the doctor that they have heart pain, the doctor will ask if they smoke and many other things that a regular checkup doesn't.
To conclude, a domain understanding of the data is important before you think your data has a class imbalance. When you do know the class imbalance, you can roughly estimate how imbalanced it is.
