Sometimes we come across datasets where classes are imbalanced. For example, class A may have 2000 instances, but class B has only 200. How can we train a classifier for such datasets?

  • $\begingroup$ cross validation $\endgroup$ – Sreejithc321 Jun 26 '15 at 4:16

Well, you just train it on unbalanced dataset, it is not a problem. I don't think you need to apply some special techniques.

The only case where you may want to do something special is if your classes are skewed by their nature (if the skewness is the property of class itself, not of just your specific dataset). For example, if you build a classifier which will tell if someone has a malignant tumor in X-ray image or not. By nature of the task absolute majority of patients will not have a malignant tumor. In such case you may want to tune your classifier algorithm a bit, for example introduce a weighting into your SVM or smth like it.

A special case is if you have EXTREMELY unbalanced classes. For example 100000 of positive examples and 20 negatives. In such case you will want to go away from classification task to the approach called 'anomaly detection'.

  • $\begingroup$ What do you mean by swm? $\endgroup$ – jithinjustin Jun 26 '15 at 5:00
  • $\begingroup$ Support vector machine. Just as an example, this algorithm offers an ability to assign different weights to classes manually $\endgroup$ – Maksim Khaitovich Jun 28 '15 at 15:49

There are a few of other approaches you can take to try to balance your class distribution.


Subsample Majority Class

You can balance the class distributions by subsampling the majority class.

Oversample Minority Class

Sampling with replacement can be used to increase your minority class proportion.

Adding Noise

A more sophisticated scheme is to add Gaussian, or other suitable, noise to the existing instances of the minority class in an effort to create a greater number of representative, but diverse, instances.


A popular method to synthesise minority class instances with greater complexity than pure noise addition is SMOTE ( Synthetic Minority Oversampling TEchnique ). This uses a K-member neighbourhood in feature space to impute new instances.

WEKA has a filter for this.

Though there is some evidence that this technique is not overly beneficial with high dimensional data here.

  • $\begingroup$ Can you please cite some references on the "adding noise" solution you are suggesting for example in the context of time series? $\endgroup$ – tagoma Jul 9 '17 at 20:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.