Questions tagged [class-imbalance]

Questions referring to classifiers or classifying problems where some of the classes in the data are under-represented.

Filter by
Sorted by
Tagged with
57 votes
6 answers
15k views

Should I go for a 'balanced' dataset or a 'representative' dataset?

My 'machine learning' task is of separating benign Internet traffic from malicious traffic. In the real world scenario, most (say 90% or more) of Internet traffic is benign. Thus I felt that I should ...
  • 673
38 votes
6 answers
41k views

Unbalanced multiclass data with XGBoost

I have 3 classes with this distribution: Class 0: 0.1169 Class 1: 0.7668 Class 2: 0.1163 And I am using xgboost for ...
  • 545
35 votes
4 answers
15k views

Quick guide into training highly imbalanced data sets

I have a classification problem with approximately 1000 positive and 10000 negative samples in training set. So this data set is quite unbalanced. Plain random forest is just trying to mark all test ...
  • 5,314
25 votes
4 answers
38k views

macro average and weighted average meaning in classification_report

I use the "classification_report" from from sklearn.metrics import classification_report in order to evaluate the imbalanced binary classification ...
  • 1,636
25 votes
3 answers
26k views

How do you apply SMOTE on text classification?

Synthetic Minority Oversampling Technique (SMOTE) is an oversampling technique used in an imbalanced dataset problem. So far I have an idea how to apply it on generic, structured data. But is it ...
  • 369
18 votes
4 answers
20k views

Macro- or micro-average for imbalanced class problems

The question of whether to use macro- or micro-averages when the data is imbalanced comes up all the time. Some googling shows that many bloggers tend to say that micro-average is the preferred way ...
  • 283
17 votes
4 answers
22k views

Train/Test Split after performing SMOTE

I am dealing with a highly unbalanced dataset so I used SMOTE to resample it. After SMOTE resampling, I split the resampled dataset into training/test sets using the training set to build a model and ...
  • 2,525
16 votes
4 answers
2k views

What are the implications for training a Tree Ensemble with highly biased datasets?

I have a highly biased binary dataset - I have 1000x more examples of the negative class than the positive class. I would like to train a Tree Ensemble (like Extra Random Trees or a Random Forest) on ...
  • 418
15 votes
3 answers
20k views

When should we consider a dataset as imbalanced?

I'm facing a situation where the numbers of positive and negative examples in a dataset are imbalanced. My question is, are there any rules of thumb that tell us when we should subsample the large ...
  • 574
14 votes
2 answers
4k views

Why do we need to handle data imbalance?

I would like to know why we need to deal with data imbalance. I know how to deal with it and different methods to solve the issue - by up sampling or down sampling or by using SMOTE. For example, if I ...
  • 461
13 votes
6 answers
12k views

Deep network not able to learn imbalanced data beyond the dominant class

I have data with 5 output classes. The training data has the following no of samples for these 5 classes: [706326, 32211, 2856, 3050, 901] I am using the following keras (tf.keras) code: <...
  • 251
13 votes
3 answers
21k views

Unbalanced classes -- How to minimize false negatives?

I have a dataset that has a binary class attribute. There are 623 instances with class +1 (cancer positive) and 101,671 instances with class -1 (cancer negative). I've tried various algorithms (Naive ...
11 votes
3 answers
22k views

How can I perform stratified sampling for multi-label multi-class classification?

I am asking this question for few reasons: The dataset in hand is imbalanced I used below code ...
11 votes
1 answer
5k views

Why doesn't class weight resolve the imbalanced classification problem?

I know that in imbalanced classification, the classifier tends to predict all the test labels as larger class label, but if we use class weight in loss function, it would be reasonable to expect the ...
11 votes
1 answer
5k views

Cross validation for highly imbalanced data with undersampling

In my problem, I am dealing with a highly imbalanced data set, say for every positive class there are 10000 negative one. A normal starting method to train a model is to undersample the data. In this ...
11 votes
2 answers
1k views

When do we say that the dataset is not classifiable?

I have many times analysed a dataset on which I could not really do any sort of classification. To see whether I can get a classifier I have usually used the following steps: Generate box plots of ...
  • 178
10 votes
1 answer
38k views

How does class_weights work in RandomForestClassifier

I'm facing a problem with unbalanced classes, and have tried out a couple of methods like over and under sampling. However, my cross validation mean comes out to be only 0.4 and my confusion matrix ...
  • 423
9 votes
2 answers
22k views

weighted cross entropy for imbalanced dataset - multiclass classification

I am trying to classify images to more then a 100 classes, of different sizes ranged from 300 to 4000 (mean size 1500 with std 600). I am using a pretty standard CNN where the last layer outputs a ...
  • 129
9 votes
1 answer
7k views

CNN - imbalanced classes, class weights vs data augmentation

I have a dataset with a few strongly imbalanced classes, eg. the smallest class is about 54 times smaller than the largest. Therefore, data augmentation in order to equalize the size of classes seems ...
  • 155
8 votes
1 answer
2k views

Why you shouldn't upsample before cross validation

I have an imbalanced dataset and I am trying different methods to address the data imbalance. I found this article that explains the correct way to cross-validate when oversampling data using SMOTE ...
  • 345
8 votes
1 answer
12k views

Differences between class_weight and scale_pos weight in LightGBM

I have a very imbalanced dataset with the ratio of the positive samples to the negative samples being 1:496. The scoring metric is the f1 score and my desired model is LightGBM. I am using the sklearn ...
8 votes
1 answer
1k views

What is the best performance metric used in balancing dataset using SMOTE technique

I used smote technique to oversample my dataset and now I have a balanced dataset. The problem I faced is that the performance metrics; precision, recall, f1 measure, accuracy in the imbalanced ...
8 votes
1 answer
277 views

Categorization of approaches to deal with imbalanced classes

What is the best way to categorize the approaches which have been developed to deal with imbalance class problem? This article categorizes them into: Preprocessing: includes oversampling, ...
  • 1,215
8 votes
2 answers
93 views

Which classification algorithms are negatively affected by class imbalances?

I've seen a few posts and papers floating around the web (mostly those related to over/undersampling, SMOTE, and cost-sensitive training) that, when discussing class imbalance, specify that certain ...
8 votes
1 answer
2k views

How does exactly class_weight in Keras work?

I'm working on a multi-label problem in Keras, using binary-cross-entropy loss function with sigmoid activation. Let's say I have 4 classes, so a response might look like this: ...
8 votes
1 answer
7k views

Using class weights in Keras with multiple binary outputs which are not simply one-hot-encoded

My labels are binary vectors of length 5, e.g., [0, 0, 1, 1, 1]. My label set is very biased, 1-to-50, where the case [0, 0, 0, 0, 0] is very common while all ...
8 votes
4 answers
6k views

Unbalanced class: class_weight for ML algorithms in Spark MLLib

In python sklearn, there are multiple algorithms (e.g. regression, random forest ... etc.) that have the class_weight parameter to handle unbalanced data. However, I do not find such parameter for ...
  • 2,525
7 votes
2 answers
10k views

Why will the accuracy of a highly unbalanced dataset reduce after oversampling?

I have created a synthetic dataset, with 20 samples in one class and 100 in the other, thus creating an imbalanced dataset. Now the accuracy of classification of the data before balancing is 80% while ...
  • 1,151
7 votes
4 answers
8k views

Bad classification performance of logistic regression on imbalanced data in testing as compared to training

I am trying to fit a logistic regression model to an imbalanced dataset (0.5/99.5) with high dimensionality(about 15k). I used random forest to select top 200 important features. Observations are ...
  • 131
7 votes
3 answers
2k views

For imbalanced classification, should the validation dataset be balanced?

I am building a binary classification model for imbalanced data (e.g., 90% Pos class vs 10% Neg Class). I already balanced my training dataset to reflect a a 50/50 class split, while my holdout (...
7 votes
2 answers
3k views

How to fix class imbalance in training sample?

I was very recently asked in a job interview about solutions to fix an imbalance of classes in the training dataset. Let's focus on a binary classification case. I offered two solutions: oversampling ...
7 votes
3 answers
13k views

Weighted Binary Cross Entropy Loss -- Keras Implementation

I have a binary segmentation problem with highly imbalanced data such that there are almost 60 class zero samples for every class one sample. To address this issue, I coded a simple weighted binary ...
7 votes
1 answer
5k views

Overfitting for minority class after SMOTE w/ random forests

I used SMOTE to make a predictive model, with class 1 having 1800 samples and 35000+ of class 0 samples. Hence, as per SMOTE, synthetic samples were created and the random forest was trained. However,...
  • 423
7 votes
1 answer
4k views

Class weighting during validation in Keras

I would like to know if the class weighting is also used in evaluating the loss in the validation data during the training. If not, is there a way to adjust the fit() function so that it takes into ...
  • 306
7 votes
1 answer
10k views

Imbalanced dataset in MLP classifier in python

I am dealing with imbalanced dataset and I try to make a predictive model using MLP classifier. Unfortunately the algorithm classifies all the observations from test set to class "1" and hence the f1 ...
7 votes
4 answers
1k views

Training and testing AdaBoost for low probability classification

I have a dataset that I want to classify as fraud/not fraud and I have many weak learners. My concern is that there is much more fraud than not fraud, so my weak learners perform better than average, ...
  • 71
6 votes
2 answers
10k views

Why class weight is outperforming oversampling?

I am applying both class_weight and oversampling (SMOTE) techniques on a multiclass classification problem and getting better results when using the class_weight technique. Could someone please ...
  • 581
6 votes
2 answers
4k views

Kappa near to 60% in unbalanced (1:10) data set

As mentioned before, I have a classification problem and unbalanced data set. The majority class contains 88% of all samples. I have trained a Generalized Boosted Regression model using ...
  • 5,314
6 votes
4 answers
341 views

Is an $F_1$ score of 0.1 always bad?

I'm currently building a model to predict early mortgage delinquency (60+ days delinquent within 2 years of origination) for loans originating in 2018Q1. I will eventually train out-of-time (on loans ...
  • 213
6 votes
5 answers
7k views

In a binary classification, should the test dataset be balanced?

I have a dataset with 4519 samples labeled as "1", and 18921 samples labeled as "0" in a binary classification exercise. I am well aware that during the training phase of a classification algorithm (...
  • 173
6 votes
2 answers
3k views

Is There a Way to Re-Calibrate Predicted Probabilities After Using Class Weights?

I have classification data with far more negative instances than positive instances. I have used class weights in my models and have achieved the discrimination I want but the predicted probabilities ...
6 votes
2 answers
255 views

Can we specify the number of data generated(minority class) using SMOTE?

I am trying to improve classification of imbalanced dataset creditcard fraud using SMOTE imbalanced_learn. But, in this it generates the data to 50%, can we give a specific number for the data to be ...
6 votes
2 answers
295 views

Why real-world output of my classifier has similar label ratio to training data?

I trained a neural network on balanced dataset, and it has good accuracy ~85%. But in real world positives appear in about 10% of the cases or less. When I test network on set with real world ...
  • 63
6 votes
1 answer
821 views

Imbalanced classes (balance of train, validation, and test)

1) I am currently trying to set up a feedforward neural network with highly imbalanced classes (binary classification) in which the number of observations of class 1 is very low (and the class of ...
  • 61
6 votes
1 answer
514 views

Random Forest significantly outperforms XGBoost - problem or possible?

I have dataset of around 180k observations of 13 variables (mix of numerical and categorical features). It is binary classification problem, but classes are imbalanced (25:1 for negative ones). I ...
  • 63
6 votes
3 answers
1k views

Why did sampling boost the performance of my model?

I have an imbalanced dataset with 88 positive samples and 128575 negative samples. I was reluctant to over/undersample the data since it's a biological dataset and I didn't want to introduce synthetic ...
6 votes
3 answers
4k views

using sklearn class weight to increase number of positive guesses in extremely unbalanced data set?

Hi I have a poorly correlated and unbalanced data set I have to work with. The set is 2 classes, 0 has 96,000 values and 1 has about 200. When I run random forest or other methods I get an output like:...
6 votes
3 answers
699 views

What is the best metric to evaluate highly imbalanaced binary classifiction? (such as fraud detection in credit card)

What is the best metric to evaluate highly imbalanaced binary classifiction? (such as fraud detection in credit card? I have examining several metrics precision recall F1 lassification Report (macro ...
  • 1,636
6 votes
1 answer
397 views

How to compare two unsupervised anomaly detection algorithms on the same data-set?

I want to solve an anomaly detection problem on an unlabeled data-set. The only information about this problem is that the anomalies population is lower than 0.1%. It should be notice that the size of ...
6 votes
1 answer
2k views

How to avoid resampling part of pipeline on test data (imblearn package, SMOTE)

I am using the imblearn package to resample some data before applying other transformation/prediction techniques. Specfically, I am using SMOTE in a slightly unconventional way, as a data ...

1
2 3 4 5
11