Questions tagged [class-imbalance]

Questions referring to classifiers or classifying problems where some of the classes in the data are under-represented.

Filter by
Sorted by
Tagged with
7
votes
1answer
873 views

Why you shouldn't upsample before cross validation

I have an imbalanced dataset and I am trying different methods to address the data imbalance. I found this article that explains the correct way to cross validate when oversampling data using SMOTE ...
53
votes
6answers
14k views

Should I go for a 'balanced' dataset or a 'representative' dataset?

My 'machine learning' task is of separating benign Internet traffic from malicious traffic. In the real world scenario, most (say 90% or more) of Internet traffic is benign. Thus I felt that I should ...
34
votes
4answers
15k views

Quick guide into training highly imbalanced data sets

I have a classification problem with approximately 1000 positive and 10000 negative samples in training set. So this data set is quite unbalanced. Plain random forest is just trying to mark all test ...
22
votes
3answers
19k views

How do you apply SMOTE on text classification?

Synthetic Minority Oversampling Technique (SMOTE) is an oversampling technique used in an imbalanced dataset problem. So far I have an idea how to apply it on generic, structured data. But is it ...
32
votes
4answers
33k views

Unbalanced multiclass data with XGBoost

I have 3 classes with this distribution: Class 0: 0.1169 Class 1: 0.7668 Class 2: 0.1163 And I am using xgboost for ...
4
votes
3answers
914 views

Why did sampling boost the performance of my model?

I have an imbalanced dataset with 88 positive samples and 128575 negative samples. I was reluctant to over/undersample the data since it's a biological dataset and I didn't want to introduce synthetic ...
15
votes
3answers
15k views

Train/Test Split after performing SMOTE

I am dealing with a highly unbalanced dataset so I used SMOTE to resample it. After SMOTE resampling, I split the resampled dataset into training/test sets using the training set to build a model and ...
5
votes
2answers
2k views

Is There a Way to Re-Calibrate Predicted Probabilities After Using Class Weights?

I have classification data with far more negative instances than positive instances. I have used class weights in my models and have achieved the discrimination I want but the predicted probabilities ...
1
vote
1answer
93 views

Class imbalance strategies

When dealing with the class imbalance problem in a binary classifier, there are three ways I know of to address it: over-sampling, under-sampling and using cost-sensitive methods. Are there any ...
4
votes
2answers
1k views

For imbalanced classification, should the validation dataset be balanced?

I am building a binary classification model for imbalanced data (e.g., 90% Pos class vs 10% Neg Class). I already balanced my training dataset to reflect a a 50/50 class split, while my holdout (...
2
votes
4answers
1k views

Best methods to solve class imbalance problem and why?

I have a data set where I need to detect fraud. 99% are not fraud and 1% are. What methods can be used to solve problems where classes are imbalanced?
15
votes
3answers
18k views

When should we consider a dataset as imbalanced?

I'm facing a situation where the numbers of positive and negative examples in a dataset are imbalanced. My question is, are there any rules of thumb that tell us when we should subsample the large ...
15
votes
4answers
2k views

What are the implications for training a Tree Ensemble with highly biased datasets?

I have a highly biased binary dataset - I have 1000x more examples of the negative class than the positive class. I would like to train a Tree Ensemble (like Extra Random Trees or a Random Forest) on ...
11
votes
2answers
957 views

When do we say that the dataset is not classifiable?

I have many times analysed a dataset on which I could not really do any sort of classification. To see whether I can get a classifier I have usually used the following steps: Generate box plots of ...
9
votes
1answer
5k views

CNN - imbalanced classes, class weights vs data augmentation

I have a dataset with a few strongly imbalanced classes, eg. the smallest class is about 54 times smaller than the largest. Therefore, data augmentation in order to equalize the size of classes seems ...
8
votes
2answers
19k views

weighted cross entropy for imbalanced dataset - multiclass classification

I am trying to classify images to more then a 100 classes, of different sizes ranged from 300 to 4000 (mean size 1500 with std 600). I am using a pretty standard CNN where the last layer outputs a ...
7
votes
2answers
2k views

How to fix class imbalance in training sample?

I was very recently asked in a job interview about solutions to fix an imbalance of classes in the training dataset. Let's focus on a binary classification case. I offered two solutions: oversampling ...
4
votes
3answers
410 views

Train classifier on balanced dataset and apply on imbalanced dataset?

I have a labelled training dataset DS1 with 1000 entries. The targets (True/False) are nearly balanced. With sklearn, I have tried several algorithms, of which the GradientBoostingClassifier works ...
2
votes
1answer
85 views

How much imbalance in a training set is a problem?

In a simple binary classification problem, at what point does majority class to minority class become significant become significant? Intuitively, I would expect a 3:1 ratio to not be an issue, maybe ...
1
vote
1answer
1k views

Scripting code for class imbalance in Biolabs Orange

I'm trying to manipulate some data in Biolabs Orange, using the built in Python Script widget and information at Biolabs Orange tutorial on scripting. However, I'm struggling with taking the results ...
9
votes
1answer
4k views

Why doesn't class weight resolve the imbalanced classification problem?

I know that in imbalanced classification, the classifier tends to predict all the test labels as larger class label, but if we use class weight in loss function, it would be reasonable to expect the ...
12
votes
3answers
18k views

Unbalanced classes -- How to minimize false negatives?

I have a dataset that has a binary class attribute. There are 623 instances with class +1 (cancer positive) and 101,671 instances with class -1 (cancer negative). I've tried various algorithms (Naive ...
5
votes
3answers
28k views

SMOTE and multi class oversampling

I have read that the SMOTE package is implemented for binary classification. In the case of n classes, it creates additional examples for the smallest class. Can I balance all the classes by running ...
4
votes
3answers
5k views

How to handle "unknown" category in machine learning classification problems?

Tutorial problems come in the form of binary or mult-class classification where data are all properly labelled. In real-life applications, there are incoming data that do not belong to any category ...
4
votes
5answers
5k views

In a binary classification, should the test dataset be balanced?

I have a dataset with 4519 samples labeled as "1", and 18921 samples labeled as "0" in a binary classification exercise. I am well aware that during the training phase of a classification algorithm (...
4
votes
2answers
1k views

Oversampling before Cross-Validation, is it a problem?

I have a multi-class classification problem to solve which is highly imbalanced. Obviously I'm doing oversampling, but I'm doing cross-validation with the over-sampled dataset, as a result of which I ...
14
votes
3answers
22k views

macro average and weighted average meaning in classification_report

I use the "classification_report" from from sklearn.metrics import classification_report in order to evaluate the imbalanced binary classification ...
8
votes
1answer
1k views

What is the best performance metric used in balancing dataset using SMOTE technique

I used smote technique to oversample my dataset and now I have a balanced dataset. The problem I faced is that the performance metrics; precision, recall, f1 measure, accuracy in the imbalanced ...
7
votes
1answer
4k views

Class weighting during validation in Keras

I would like to know if the class weighting is also used in evaluating the loss in the validation data during the training. If not, is there a way to adjust the fit() function so that it takes into ...
6
votes
2answers
8k views

Why will the accuracy of a highly unbalanced dataset reduce after oversampling?

I have created a synthetic dataset, with 20 samples in one class and 100 in the other, thus creating an imbalanced dataset. Now the accuracy of classification of the data before balancing is 80% while ...
6
votes
1answer
5k views

Overfitting for minority class after SMOTE w/ random forests

I used SMOTE to make a predictive model, with class 1 having 1800 samples and 35000+ of class 0 samples. Hence, as per SMOTE, synthetic samples were created and the random forest was trained. However,...
3
votes
2answers
6k views

Imbalanced dataset: how to deal with test data?

I plan to use many methods to solve the imbalanced dataset problem on the training set. But I couldn't find any paper that describes how they dealt with the test dataset? I assume that they just ...
3
votes
2answers
10k views

Ratio of positive to negative sample in data set for best classification

Suppose I have 100 positive samples. How many negative samples do I need to have in order to make the classifier work the best. In many papers, I have noticed that they take 4 times or 5 times the ...
3
votes
4answers
2k views

How to learn a classifier from a dataset with high imbalance

What are the most useful techniques for learning a binary classifier from a dataset with a high degree of imbalance (i.e., a dataset with the "target" class being much rarer than the "background" ...
2
votes
3answers
3k views

Handling large imbalanced data set

I have an imbalanced data set consisting of some 10's of millions text strings, each with thousands of features created by uni- and bigrams, and additionally I have also the string length and entropy ...
2
votes
4answers
973 views

How to deal with class imbalance in a neural network?

Suppose we have a game and its action space contains two possible actions: A and B. We have a labelled dataset of state-action ...
1
vote
1answer
550 views

ROC AUC score is better if test data is imbalanced

I have an imbalanced dataset and I'm using XGBoost to do binary classification. I used down sampling together with target and one hot encoding for train data. For ...
4
votes
1answer
930 views

Overfitting - how to detect it and reduce it?

I have a side project where I am doing credit scoring using R (sample size around 16k for train data and 4k for test data, and also another two 20k data batches for out-of-time validation) with ...
3
votes
2answers
202 views

High Recall but too low Precision result in imbalanced data

I was training a model using XGBoost Classifier on a heavy imbalanced database with 232:1 of binary class. Because my training data contains 750k rows and 320 features (after doing many feature ...
2
votes
2answers
107 views

Undersampling for credit card fraud detection before or after Train/Test Split

I have a credit card dataset with 98% transactions are Non-Fraud and 2% are fraud. I have been trying to undersample the majotrity class before train and test split and get very good recall and ...
2
votes
0answers
84 views

intercept correction in deep learning

Say I have an imbalanced data set, and I decided to over/undersample it during model training. I would then like to predict on new records but using the original, true imbalance in the classes as an ...
2
votes
2answers
76 views

Is the PR AUC invariant under label flip?

The ROC-AUC curve is invariant under a flip of the labels. I don't know if its a famous result so I will give the proof below. My question is if the PR-AUC curve also has this property. I have not ...
1
vote
0answers
1k views

Can SMOTE be applied over sequence of words (sentences)?

I have a highly unbalanced text classification data. I am trying to over-sample through SMOTE. I have a doubt that applying SMOTE over sequence of word indices will give me valid data points or not (...
0
votes
1answer
140 views

How does class_weight work in Decision Tree?

I am interested in Cost-Sensitive learning. And I am trying to understand how class_weight in DecisionTree works in terms of math. I read a lot of articles that ...
6
votes
1answer
315 views

How to compare two unsupervised anomaly detection algorithms on the same data-set?

I want to solve an anomaly detection problem on an unlabeled data-set. The only information about this problem is that the anomalies population is lower than 0.1%. It should be notice that the size of ...
3
votes
1answer
49 views

The most informative curve for imbalance datasets

For the imbalanced datasets: Can we say the Precision-Recall curve is more informative, thus accurate, than ROC curve? Can we rely on F1-score to evaluate the skillfulness of the resulted model in ...
3
votes
2answers
145 views

Does Sampling size matters in Multi classification Model

I am working on a multi class classification model where few of the class are with less data compare to other classes. I used random sampling technique to create a sample from the population keeping ...
2
votes
1answer
27 views

On what threshold we should resample the data?

When working with churn datasets, we usually find imbalanced datasets. My question is how to decide on what basis we should resample the data. For example: while splitting the data before training we ...
2
votes
2answers
46 views

Doesn't over(/under)sampling an imbalanced dataset cause issues?

I'm reading a lot about how to use different metrics specifically for imbalanced datasets (e.g. two classes present, but 80% of the data is one class) and how to tackle the issue of imbalanced ...
2
votes
1answer
280 views

SMOTE oversampling for class imbalanced dataset introduces bias in final distribution

I have a problem statement where percentage of goods (denoted by 0) is 95%, and for bads (denoted by 1) it is 5% only. One way is to do under sampling of goods so that model understands the patterns ...