What is the best way to categorize the approaches which have been developed to deal with imbalance class problem?

This article categorizes them into:

  1. Preprocessing: includes oversampling, undersampling and hybrid methods,
  2. Cost-sensitive learning: includes direct methods and meta-learning which the latter further divides into thresholding and sampling,
  3. Ensemble techniques: includes cost-sensitive ensembles and data preprocessing in conjunction with ensemble learning.

The second classification:

  1. Data Pre-processing: includes distribution change and weighting the data space. One-class learning is considered as distribution change.
  2. Special-purpose Learning Methods
  3. Prediction Post-processing: includes threshold method and cost-sensitive post-processing
  4. Hybrid Methods:

The third article:

  1. Data-level methods
  2. Algorithm-level methods
  3. Hybrid methods

The last classification also considers output adjustment as an independent approach.

Thanks in advance.

  • 4
    $\begingroup$ The very short answer: all of them are the best and all of them are the worst! Classification and data mining in general are very context sensitive. There is no one size fit all solution in this domain. By the way, the best approach, in very generic terms, is usually a combination of the best decisions at different levels from the feature extraction, to the evaluation scheme. $\endgroup$ Jun 12, 2018 at 23:53
  • $\begingroup$ @mok Thanks. Could you please let me know the class-weight in the sklearn's classifiers e.g., logistic regression is classified into which category? $\endgroup$
    – ebrahimi
    Jun 13, 2018 at 19:06
  • $\begingroup$ @ebrahimi, it should fall into algorithm level because only the weights are adjusted according to a passed dictionary or calculated (inferred) according to the values of y (class) and the data remains untouched. $\endgroup$ Jun 18, 2018 at 7:07
  • $\begingroup$ @SanjayKrishna Thanks a lot. In case of the first categorization, it falls into cost-sensitive learning, doesn't it? Also, in case of the second taxonomy, it would be classified into the third category i.e., cost-sensitive post-processing. is it true? The second answer to this: stackoverflow.com/questions/32492550/… is also useful. $\endgroup$
    – ebrahimi
    Jun 18, 2018 at 7:52

1 Answer 1


The way I see it all three categorizations agree in many things. For example, all three have a category for pre-processing steps.

I would tend to mostly agree on the third categorization as its more generic and encompasses more things.

  • The data-level category includes any pre-processing steps dealing with class imbalance (e.g. over/under sampling).
  • The algorithm-level could be considered to include the second categories of the first two articles. Any change to the algorithm that deals with class imbalance would go here (e.g. class weighting).
  • Finally, a hybrid category for combining the two.

The only thing missing from the first two articles are the post-processing steps, which to be honest, aren't used in practice as often as the other.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.