Questions tagged [class-imbalance]
Questions referring to classifiers or classifying problems where some of the classes in the data are under-represented.
516
questions
0
votes
0
answers
8
views
Understanding Cost-sensitive Decision Trees Behaviour
I have a binary classification problem with imbalanced data and am attempting to use cost-sensitive learning to handle the imbalance. I have used LogisticRegression, LinearSVC, SVC and DecisionTree ...
- 355
0
votes
0
answers
14
views
Determining the information loss due to undersampling
I have an image dataset that I need to segment into directories (train, validation and test) using ImageDataGenerator in TensorFlow/Keras. The dataset is highly imbalanced:
For this I have decided to ...
0
votes
0
answers
4
views
how to find documents similar to a predefined set of documents
From big population of documents I would like to find those similar to a predefined set of documents.
All documents inside the set are similar to each other, but very few documents from the population ...
0
votes
0
answers
12
views
Early stopping on validation loss or macro-F1?
I am working on an extremely imbalanced dataset to build a classification model. The number of classes is 53 classes. I use early stopping on the validation loss to prevent the model from overfitting.
...
- 252
1
vote
1
answer
15
views
Do Sampling before or after TFIDF step?
This is a multiclass text classification problem. The dataset has a class imbalance and I'm planning to use a sampling technique before modeling.
Should the sampling be done before/after the ...
- 111
3
votes
1
answer
252
views
Why is Data with an Overrepresented Class called Imbalanced not Unbalanced?
I've seen the term Imbalanced used to described data that has an over-representation of one class.
What's the reasoning behind naming this type of data Imbalanced as opposed to Unbalanced, which seems ...
- 175
0
votes
0
answers
22
views
Classification to predict imbalanced dataset with rare event?
I'm dealing with a dataset to train a model to predict the number occurrences of events (probability(event1), probability(event2),etc). I have over 400 features. However, as shown in the table below, ...
- 101
4
votes
2
answers
494
views
Flipping the labels in a binary classification gives different model and results
I have an imbalanced dataset and I want to train a binary classifier to model the dataset.
Here was my approach which resulted into (relatively) acceptable performance:
1- I made a random split to get ...
- 43
0
votes
0
answers
12
views
Do we need to apply weights if we are using the AUC PR for imbalanced data?
If we want to apply a classification model to imbalanced data we should
Use a cost matrix or apply weights.
Use the AUC PR curve instead of the Accuracy metric to optimize my model (or the AUC ROC).
...
- 151
0
votes
0
answers
24
views
Downsampling in sklearn. Test and Train performance question
I have a class imbalanced data set, and have the following set up to handle class imbalance. I first split to test and train and only perform downsampling on the training set and then get the test ...
- 121
0
votes
0
answers
22
views
Feature Importance and Threshold Moving
Problem Type : Binary Classification
Dataset : Imbalanced
Current sklearn pipeline uses XGBoost model and involves moving threshold from 0.5 to a considerably higher value like 0.8 - 0.9.
Is it viable ...
- 1
1
vote
1
answer
47
views
Why is my model overfitting?
I am building a classification model based on some machine performance data. Unfortunately for me, it seems to over-fit no matter what I change. The dataset is quite large so I'll share the final ...
1
vote
1
answer
40
views
Why is gradient boosting better than random forest for unbalanced data?
I've searched everywhere and still couldn't figure this one out.
This post mentioned that Gradient Boosting is better than Random Forest for unbalanced data. Why is that? Is Random Forest worse ...
0
votes
0
answers
37
views
Different training score but same test score when using pipeline
I have a problem that produce different training score when using pipeline and manual.
MANUAL :
...
0
votes
0
answers
36
views
Oversampling SMOTE sampling strategy ratio
I have 36168 data with imbalanced target. 88,3% is 0 (31970 data) and 11,7% is 1 (4198 data).
I want to apply oversampling using SMOTE. Is it ideal to make it the same amount of data so the 0 & 1 ...
0
votes
0
answers
78
views
AUC OvO vs AUC OvR vs F1-Score in Multiclass Model Selection, what is better?
Given a multiclass classification task, I am looking at the best metric between AUC OvO, AUC OvR and F1-Score score to compare models.
The class distribution is the following (there is no 'class ...
0
votes
0
answers
15
views
Which model to use for multitarget classification with strong class imbalance and many categorical variables?
I have a small dataset 79 observations in 21 variables.
Almost all the variables are categorical variables in the format yes/no or 1/2/3.
I would like to predict jointly three of these variables ...
0
votes
1
answer
92
views
Plotting a no-skill model in a precision-recall curve
I am following this tutorial to apply threshold tuning using precision-recall curve for an imbalanced dataset
Within the tutorial, a no-skill model is defined as:
A no-skill model is represented by a ...
- 355
0
votes
1
answer
51
views
How can I say if I have a class imbalance issue in my data?
Assuming I have this dataset:
Label --- %Total
0 -------- 18.53%
1 -------- 8.18%
2 -------- 26.22%
3 -------- 16.46%
4 -------- 8.62%
5 -------- 9.58%
6 -------- 5.88%
7 -------- 6.53%
I could say I ...
0
votes
0
answers
14
views
Why we cannot calculate an ROC curve in cost sensitive learning?
In the Applied Predictive Modeling book, cost sensitivity learning approach, the author(s) write:
One consequence of this approach is that class probabilities cannot be
generated for the model, at ...
0
votes
0
answers
29
views
Model with no historical data
I need to develop a new credit default classification model for which there are a lot of features available but very few historical data (because it's a new activity launched by the company I work for)...
- 161
2
votes
1
answer
58
views
Why does prediction calibration on a resampling mode does not meet the expectation?
I am doing a small project to predict the write-off probability of our defaulted customers.
In the original population, the write-off rate is about 0.515. Now, for some reason I have to undersample ...
- 21
0
votes
1
answer
60
views
Classification on severe Class Imbalance high dimensional data
Dear DataScience Community, I am working on class imbalance tabular data with high-dimension inputs. The tabular data is derived from the satellite data pixels, and I have inflated the train data ...
- 101
3
votes
2
answers
103
views
Precision, recall and importance of them in the imbalance problem
I have a test dataset. The dataset is an imbalanced dataset. The total training instances for the dataset is 543 among them minority class(yes) is 75 and the majority class(No) is 468. The class of ...
- 359
2
votes
2
answers
333
views
How to calculate accuracy of an imbalanced dataset
I like to understand what is the accuracy of an imbalanced dataset.
Let's suppose we have a medical dataset and we want to predict the disease among the patients. Say, in an existing dataset 95% of ...
- 359
0
votes
1
answer
30
views
Top N accuracy for an imbalanced multiclass classification problem
I have a multi-class classification problem that is imbalanced. The task is about animal classification.
Since it's imbalanced, I am using macro-F1 metric and the current result that I have is: ...
- 252
0
votes
0
answers
41
views
The paradox of Imbalanced binary classification ¿To do something or to do nothing?
Context: Suppose we are interested in deploy a machine learning model to solve a problem of binary classification; furthermore, assume that the dataset $\mathcal{D}$ for the training of our model ...
- 123
1
vote
1
answer
27
views
scale_pos_weight effect in XGBClassifier
I can't find a satisfactory explanation about the effect of scale_pos_weight on an XGBClassifier.
It says everywhere to set it to Count of negatives / Count of ...
- 161
0
votes
1
answer
19
views
Do I need to use AUPRC for reporting classification results on an imbalanced dataset when the model was trained using upsampling and CV
I am working on a binary classification problem which dataset has about 5% of positive class samples. I split the dataset, 70% for training and 30% for testing. I used the test data only once for ...
- 1
0
votes
0
answers
31
views
the alpha balance parameter in focal loss
I want to use a weighted focal loss for my imbalanced object detection problem.
let's say I have 3 classes, with 10000, 1000, and 100 examples each.
The focal loss function looks like this:
\begin{...
0
votes
1
answer
41
views
How to effectively evaluate a model with highly imbalanced and limited dataset
Most data imbalance questions on this stack have been asking How to learn a better model, but I tend to think one other problem is How do we define "better" (i.e. fairly evaluate the learned ...
- 13
0
votes
0
answers
7
views
Lift - Class ratio as actual randomness-measure
Context
The Lift should show how a machine learning model performs better than randomness. Thus, a curve representing the ratio between the predicted class of a ...
- 111
0
votes
0
answers
20
views
How to choose a machine learning sampling method?
I have a multiclassification problem with a training dataset of 3 groups (50 samples:150 samples: 100 samples).
I have tried comparing models running SMOTE oversampling and class weighting (using ...
- 1
0
votes
0
answers
45
views
Data Imbalance in Contextual Bandit with Thompson Sampling
I'm working with the Online Logistic Regression Algorithm (Algorithm 3) of Chapelle and Li in their paper, "An Empirical Evaluation of Thompson Sampling" (https://papers.nips.cc/paper/2011/...
- 1
0
votes
0
answers
116
views
Using class weights with training on imbalanced dataset gives worse result w.r.t logloss than without weights
I am trying to make a model for usual binary classification that is able to predict probabilities of classes. I have not very big dataset of 10k objects where classes are imbalanced as 80:20 and tried ...
3
votes
0
answers
92
views
Zero-shot learning for tabular data?
Can anyone point me to methods for zero-shot learning on tabular data? There is some very cool work being done for zero-shot learning on images and text, but I'm struggling to find work being done to ...
- 63
0
votes
0
answers
20
views
Sampling Highly Imbalanced Large Dataset
I am working on a model which will run monthly on 8M users. I've snapshot-wise data in training set, eg:
Jan, 21 Snapshot : 8M Total : 233 Positives Rest Negative
Feb, 21 Snapshot : 8M Total : 599 ...
2
votes
1
answer
258
views
class weighted classification
I am working on my multi-class classification project and I have a question:
I have three classes in proportion: 50%, 47% and 3%. I decided to use ...
- 41
0
votes
1
answer
120
views
ROC-AUC Imbalanced Data Score Interpretation
I have a binary response variable (label) 𝐵 in a dataset with around 50,000 observations.
The training set is somewhat imbalanced with, 𝐵𝑖=1 making up about 33% of the observation's and 𝐵𝑖=0 ...
- 21
0
votes
0
answers
173
views
under sample to get specific number of samples per class using tomek links of imblearn
I have a dataset with classes in my target column distributed like shown below.
...
1
vote
0
answers
73
views
Can data augmentation techniques be misleading?
In an attempt to handle imbalance in data, especially in the case of extremely imbalanced data, can the various data augmentation techniques create some bias?
- 23
3
votes
1
answer
111
views
Area Under the Precision Recall Curve
I have got the following Precision Recall Curve for a classifier I built using AutoML. Most of the Precisio-Recall curves tend to start from (0, 1) go towards (1,0). But mine is the opposite. But I ...
- 151
0
votes
0
answers
38
views
Complex balanced dataloading from multiple imbalanced datasets?
The Setting
Let's suppose that I have an imbalanced dataset.
For training purposes, I want to implement a dataloading scheme that samples from this dataset in a more balanced way.
I want to leverage ...
- 327
0
votes
0
answers
21
views
Can you get a very good AUC-ROC score despite predicting all rows to have the same probability?
On the test set of a binary classification problem, the p25, p50 and p75 of the predictions are very close to each other (e.g. 0.123).
Is it possible that my model can achieve a high AUC-ROC (e.g. 0....
- 1
1
vote
1
answer
24
views
Does synthetic data be over sampled as well?
I'm building a binary text classifier, the ratio between the positives and negatives is 1:100 (100 / 10000).
By using back translation as an augmentation, I was able to get 400 more positives.
Then I ...
0
votes
0
answers
28
views
Oversampling in imbalanced classification producing perfect classification
I have a 40 to 1 imbalance for my binary classification problem. I proceeded to solve the imbalance by oversampling and I generated synthetic samples to oversample my minority class. After having ...
1
vote
1
answer
63
views
How to define minority/majority class in a multi-classification task
I am studying classification in imbalanced datasets and I am learning under/over sampling strategies as a way to address the issue. While the literature agrees one needs to oversample 'minority' ...
- 111
0
votes
1
answer
77
views
Imbalanced data: understanding example from Bishop PRML book?
I'm trying to understand the 3-step procedure to compensate for the effects of imbalanced data described in Section 1.5.4 - pg 45 of Bishop's PRML book. Please refer to the following excerpt from the ...
- 3
1
vote
1
answer
103
views
Class imbalance: Will transforming multi-label (aka multi-task) to multi-class problem help?
I noticed this and this questions, but my problem is more about class imbalance. So now I have, say, 1000 targets and some input samples (with some feature vectors). Each input sample can have label ...
- 13
1
vote
1
answer
23
views
Will a classifier trained on undersampled data make accurate predictions on new imbalanced data?
I have a dataset with about 200,000 entries. The target variable is binary, and only 4,000 instances belong to the class of interest.
I would like to undersample the majority class so that we have a ...
- 85