1
$\begingroup$

I'm facing a problem about making a classification on a dataset. The target variable is binary (with 2 classes, 0 and 1). I have 8,161 samples in the training dataset. And for each class, I have:

  • class 0: 6,008 samples, 73.6% of total numbers.
  • class 1: 2,153 samples, 26.4%

My questions are:

  • In this case, should I consider the dataset I used as an imbalanced dataset?

  • If it was, should I process the data before using RandomForest to make a prediction?

  • If it was not an imbalanced dataset, could somebody tell me in which situation (like what ratio for each class) I could consider a dataset as imbalanced?

$\endgroup$
2

3 Answers 3

1
$\begingroup$

Intuitively, it seems like an imbalanced dataset to have ~75/25 ratio of class labels.

If you want to take a look at it theoretically, you can do a hypothesis test. For a sample size of 8161, you can assume that the dataset is 50/50 as null hypothesis and then compute the probability that a number extreme as 6008 or more of them belong to one class as p-value and then try to reject the null hypothesis if the p value is low (less than 0.05 or 0.01 as per choice.)

This can be done using a binomial distribution.

$\endgroup$
0
$\begingroup$

I think you can speak of imbalanced targets if (in case of a binary classification problem) the classes are not represented in a 50:50 manner. This is almost always the case.

With about 25/75 in your case, I would see this as „imbalanced“. There are some strategies to deal with this problem, such as (re)sampling so that you achieve a 50:50 balanced sample (essentially you will lose observations in the majority class here). Alternatively you can use synthetic oversampling (SMOTE) and related rechniques.

However, some packages come with built-in options to deal with unbalanced targets, e.g. sklearn‘s random forest (option class_weight). Check the docs.

$\endgroup$
1
  • $\begingroup$ This is bad advice. You should never over sample on unbalanded datasets. Only under-representitive. $\endgroup$
    – GooJ
    Sep 10, 2022 at 16:11
0
$\begingroup$

You can try ydata-profiling (https://github.com/ydataai/ydata-profiling). There's a property that measure whether a class is imbalanced or not based on entropy, might be helpful.

https://github.com/ydataai/ydata-profiling/blob/master/src/ydata_profiling/model/pandas/imbalance_pandas.py

The concept to validate imbalanced classes is pretty straightforward - on a dataset of n instances, if you have k classes of size Ci you can compute Shanon Entropy as follows:

It is one of the most precise metrics I've found, to validate whether the dataset is imbalanced, given Shanon-Entropy is commonly used to measure the impurity or uncertainty within a set of data.

$\endgroup$
3
  • $\begingroup$ Do you mind to point to us specifically which property it is? There are quite a lot of things in this project. $\endgroup$
    – lpounng
    May 16, 2023 at 10:00
  • $\begingroup$ As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    May 21, 2023 at 8:00
  • $\begingroup$ Just updated the details :) $\endgroup$
    – FabC
    May 22, 2023 at 1:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.