Questions tagged [class-imbalance]

Questions referring to classifiers or classifying problems where some of the classes in the data are under-represented.

Filter by
Sorted by
Tagged with
53
votes
6answers
14k views

Should I go for a 'balanced' dataset or a 'representative' dataset?

My 'machine learning' task is of separating benign Internet traffic from malicious traffic. In the real world scenario, most (say 90% or more) of Internet traffic is benign. Thus I felt that I should ...
34
votes
4answers
15k views

Quick guide into training highly imbalanced data sets

I have a classification problem with approximately 1000 positive and 10000 negative samples in training set. So this data set is quite unbalanced. Plain random forest is just trying to mark all test ...
32
votes
4answers
34k views

Unbalanced multiclass data with XGBoost

I have 3 classes with this distribution: Class 0: 0.1169 Class 1: 0.7668 Class 2: 0.1163 And I am using xgboost for ...
22
votes
3answers
20k views

How do you apply SMOTE on text classification?

Synthetic Minority Oversampling Technique (SMOTE) is an oversampling technique used in an imbalanced dataset problem. So far I have an idea how to apply it on generic, structured data. But is it ...
15
votes
4answers
2k views

What are the implications for training a Tree Ensemble with highly biased datasets?

I have a highly biased binary dataset - I have 1000x more examples of the negative class than the positive class. I would like to train a Tree Ensemble (like Extra Random Trees or a Random Forest) on ...
15
votes
3answers
16k views

Train/Test Split after performing SMOTE

I am dealing with a highly unbalanced dataset so I used SMOTE to resample it. After SMOTE resampling, I split the resampled dataset into training/test sets using the training set to build a model and ...
15
votes
4answers
15k views

Macro- or micro-average for imbalanced class problems

The question of whether to use macro- or micro-averages when the data is imbalanced comes up all the time. Some googling shows that many bloggers tend to say that micro-average is the preferred way ...
15
votes
3answers
24k views

macro average and weighted average meaning in classification_report

I use the "classification_report" from from sklearn.metrics import classification_report in order to evaluate the imbalanced binary classification ...
15
votes
3answers
18k views

When should we consider a dataset as imbalanced?

I'm facing a situation where the numbers of positive and negative examples in a dataset are imbalanced. My question is, are there any rules of thumb that tell us when we should subsample the large ...
15
votes
2answers
3k views

Why do we need to handle data imbalance?

I would like to know why we need to deal with data imbalance. I know how to deal with it and different methods to solve the issue - by up sampling or down sampling or by using SMOTE. For example, if I ...
12
votes
3answers
18k views

Unbalanced classes -- How to minimize false negatives?

I have a dataset that has a binary class attribute. There are 623 instances with class +1 (cancer positive) and 101,671 instances with class -1 (cancer negative). I've tried various algorithms (Naive ...
11
votes
3answers
19k views

How can I perform stratified sampling for multi-label multi-class classification?

I am asking this question for few reasons: The dataset in hand is imbalanced I used below code ...
11
votes
2answers
972 views

When do we say that the dataset is not classifiable?

I have many times analysed a dataset on which I could not really do any sort of classification. To see whether I can get a classifier I have usually used the following steps: Generate box plots of ...
10
votes
1answer
4k views

Cross validation for highly imbalanced data with undersampling

In my problem, I am dealing with a highly imbalanced data set, say for every positive class there are 10000 negative one. A normal starting method to train a model is to undersample the data. In this ...
10
votes
6answers
10k views

Deep network not able to learn imbalanced data beyond the dominant class

I have data with 5 output classes. The training data has the following no of samples for these 5 classes: [706326, 32211, 2856, 3050, 901] I am using the following keras (tf.keras) code: <...
9
votes
1answer
4k views

Why doesn't class weight resolve the imbalanced classification problem?

I know that in imbalanced classification, the classifier tends to predict all the test labels as larger class label, but if we use class weight in loss function, it would be reasonable to expect the ...
9
votes
2answers
20k views

weighted cross entropy for imbalanced dataset - multiclass classification

I am trying to classify images to more then a 100 classes, of different sizes ranged from 300 to 4000 (mean size 1500 with std 600). I am using a pretty standard CNN where the last layer outputs a ...
9
votes
1answer
31k views

How does class_weights work in RandomForestClassifier

I'm facing a problem with unbalanced classes, and have tried out a couple of methods like over and under sampling. However, my cross validation mean comes out to be only 0.4 and my confusion matrix ...
9
votes
1answer
5k views

CNN - imbalanced classes, class weights vs data augmentation

I have a dataset with a few strongly imbalanced classes, eg. the smallest class is about 54 times smaller than the largest. Therefore, data augmentation in order to equalize the size of classes seems ...
8
votes
1answer
1k views

What is the best performance metric used in balancing dataset using SMOTE technique

I used smote technique to oversample my dataset and now I have a balanced dataset. The problem I faced is that the performance metrics; precision, recall, f1 measure, accuracy in the imbalanced ...
8
votes
1answer
269 views

Categorization of approaches to deal with imbalanced classes

What is the best way to categorize the approaches which have been developed to deal with imbalance class problem? This article categorizes them into: Preprocessing: includes oversampling, ...
8
votes
4answers
6k views

Unbalanced class: class_weight for ML algorithms in Spark MLLib

In python sklearn, there are multiple algorithms (e.g. regression, random forest ... etc.) that have the class_weight parameter to handle unbalanced data. However, I do not find such parameter for ...
8
votes
2answers
85 views

Which classification algorithms are negatively affected by class imbalances?

I've seen a few posts and papers floating around the web (mostly those related to over/undersampling, SMOTE, and cost-sensitive training) that, when discussing class imbalance, specify that certain ...
7
votes
1answer
957 views

Why you shouldn't upsample before cross validation

I have an imbalanced dataset and I am trying different methods to address the data imbalance. I found this article that explains the correct way to cross-validate when oversampling data using SMOTE ...
7
votes
2answers
2k views

How to fix class imbalance in training sample?

I was very recently asked in a job interview about solutions to fix an imbalance of classes in the training dataset. Let's focus on a binary classification case. I offered two solutions: oversampling ...
7
votes
3answers
10k views

Weighted Binary Cross Entropy Loss -- Keras Implementation

I have a binary segmentation problem with highly imbalanced data such that there are almost 60 class zero samples for every class one sample. To address this issue, I coded a simple weighted binary ...
7
votes
1answer
4k views

Class weighting during validation in Keras

I would like to know if the class weighting is also used in evaluating the loss in the validation data during the training. If not, is there a way to adjust the fit() function so that it takes into ...
7
votes
1answer
7k views

Using class weights in Keras with multiple binary outputs which are not simply one-hot-encoded

My labels are binary vectors of length 5, e.g., [0, 0, 1, 1, 1]. My label set is very biased, 1-to-50, where the case [0, 0, 0, 0, 0] is very common while all ...
7
votes
4answers
1k views

Training and testing AdaBoost for low probability classification

I have a dataset that I want to classify as fraud/not fraud and I have many weak learners. My concern is that there is much more fraud than not fraud, so my weak learners perform better than average, ...
6
votes
2answers
8k views

Why will the accuracy of a highly unbalanced dataset reduce after oversampling?

I have created a synthetic dataset, with 20 samples in one class and 100 in the other, thus creating an imbalanced dataset. Now the accuracy of classification of the data before balancing is 80% while ...
6
votes
2answers
4k views

Kappa near to 60% in unbalanced (1:10) data set

As mentioned before, I have a classification problem and unbalanced data set. The majority class contains 88% of all samples. I have trained a Generalized Boosted Regression model using ...
6
votes
4answers
8k views

Bad classification performance of logistic regression on imbalanced data in testing as compared to training

I am trying to fit a logistic regression model to an imbalanced dataset (0.5/99.5) with high dimensionality(about 15k). I used random forest to select top 200 important features. Observations are ...
6
votes
2answers
170 views

Can we specify the number of data generated(minority class) using SMOTE?

I am trying to improve classification of imbalanced dataset creditcard fraud using SMOTE imbalanced_learn. But, in this it generates the data to 50%, can we give a specific number for the data to be ...
6
votes
1answer
7k views

Differences between class_weight and scale_pos weight in LightGBM

I have a very imbalanced dataset with the ratio of the positive samples to the negative samples being 1:496. The scoring metric is the f1 score and my desired model is LightGBM. I am using the sklearn ...
6
votes
2answers
243 views

Why real-world output of my classifier has similar label ratio to training data?

I trained a neural network on balanced dataset, and it has good accuracy ~85%. But in real world positives appear in about 10% of the cases or less. When I test network on set with real world ...
6
votes
1answer
752 views

Imbalanced classes (balance of train, validation, and test)

1) I am currently trying to set up a feedforward neural network with highly imbalanced classes (binary classification) in which the number of observations of class 1 is very low (and the class of ...
6
votes
3answers
4k views

using sklearn class weight to increase number of positive guesses in extremely unbalanced data set?

Hi I have a poorly correlated and unbalanced data set I have to work with. The set is 2 classes, 0 has 96,000 values and 1 has about 200. When I run random forest or other methods I get an output like:...
6
votes
3answers
536 views

What is the best metric to evaluate highly imbalanaced binary classifiction? (such as fraud detection in credit card)

What is the best metric to evaluate highly imbalanaced binary classifiction? (such as fraud detection in credit card? I have examining several metrics precision recall F1 lassification Report (macro ...
6
votes
1answer
5k views

Overfitting for minority class after SMOTE w/ random forests

I used SMOTE to make a predictive model, with class 1 having 1800 samples and 35000+ of class 0 samples. Hence, as per SMOTE, synthetic samples were created and the random forest was trained. However,...
6
votes
1answer
325 views

How to compare two unsupervised anomaly detection algorithms on the same data-set?

I want to solve an anomaly detection problem on an unlabeled data-set. The only information about this problem is that the anomalies population is lower than 0.1%. It should be notice that the size of ...
6
votes
3answers
2k views

Balanced Train set to predict Imbalanced Prediction set

One of the methods to address a classification predictive analysis on an imbalanced set consist on undersample the majority class (others approaches consist on: undersample the majority class, ...
6
votes
1answer
1k views

How does exactly class_weight in Keras work?

I'm working on a multi-label problem in Keras, using binary-cross-entropy loss function with sigmoid activation. Let's say I have 4 classes, so a response might look like this: ...
5
votes
2answers
7k views

Why class weight is outperforming oversampling?

I am applying both class_weight and oversampling (SMOTE) techniques on a multiclass classification problem and getting better results when using the class_weight technique. Could someone please ...
5
votes
4answers
182 views

Is an $F_1$ score of 0.1 always bad?

I'm currently building a model to predict early mortgage delinquency (60+ days delinquent within 2 years of origination) for loans originating in 2018Q1. I will eventually train out-of-time (on loans ...
5
votes
3answers
195 views

Metric for label imbalance

I'm looking for a metric that can be used to quantify how imbalanced the labels are in a dataset. I'm not looking for a strategy to solve the imbalance problem, I just want to present how imbalanced ...
5
votes
2answers
2k views

Is There a Way to Re-Calibrate Predicted Probabilities After Using Class Weights?

I have classification data with far more negative instances than positive instances. I have used class weights in my models and have achieved the discrimination I want but the predicted probabilities ...
5
votes
2answers
125 views

Dealing with unbalanced training set compared with real world data

I am in charge of a fraud detection model that prevents fraudulent users from using our solution. My model is performing great but the issue I have is that the more the model becomes performant the ...
5
votes
2answers
3k views

convert predict_proba results using class_weight in training

As my dataset is unbalanced(class 1: 5%, class 0: 95%) I have used class_weight="balanced" parameter to train a random forest classification model. In this way I penalize the misclassification of a ...
5
votes
1answer
25k views

How to do imbalanced classification in deep learning (tensorflow, RNN)?

I am trying to do binary classification of News Articles (Sports/Non-Sports) using recurrent neural net in tensorflow. The training data is highly skewed [Sports:Non-Sports::1:9]. I am using cross-...
5
votes
1answer
4k views

When should you balance a time series dataset?

I'm training a machine learning algorithm to classify up/down trends in a time series and I'm using an imbalanced feature set. It seems necessary to balance the data since the algorithm could learn a ...

1
2 3 4 5
10