Questions tagged [class-imbalance]
Questions referring to classifiers or classifying problems where some of the classes in the data are under-represented.
536
questions
0
votes
0
answers
9
views
Sampling strategies in multi-target classification
I am dealing with multi-target binary classifications (I have two targets). I need to use a sampling strategy. I have tried imblearn.pipeline but I'm getting the same error as this time when I'm ...
- 101
1
vote
1
answer
31
views
I used SMOTE-ENN to balance my dataset and it improved the performance metrics, but how can I be sure it's not overfitting?
The models were evaluated using 10-fold cross validation.
foldCount = StratifiedKFold(10, shuffle=True, random_state=1)
The models in question are XGBoost.
...
- 15
1
vote
1
answer
12
views
Class Imbalance in Dataset of Images
When dealing with an imbalanced dataset, I have been taught to oversample on only the train samples and not the entire dataset to avoid overfitting, however this was for structured text based data in ...
- 11
0
votes
0
answers
17
views
Classification problem with a numerical variable that uses a special (high) value to indicate a qualitatively different status
I have a classification problem where I need to predict an outcome based on 20+ variables, some categorical, some numerical. One of the numerical variables is 'dlast' - which is the number of days ...
- 101
0
votes
1
answer
14
views
Stratifed time series split with the same imbalance ratio
I am recently working on an imbalanced binary classification problem where the data is time ordered. I would like to validate my model using training/validation splits that have the same imbalance ...
1
vote
1
answer
21
views
Movement in cohorts
I am working on a user sales data which gets updated week over week. Based on the sales done in each week, the user is categorized in segment A, B or C. This means size of each segment could change ...
- 31
0
votes
1
answer
17
views
Question about collapsing variable and oversampling minority classes
i have imbalanced data consisting of nine classes, and i am planning to collapse them into two classes. i performed stratified (proportionate) sampling between test, validation, and training sets ...
- 121
0
votes
1
answer
23
views
Can I use macro recall to check if my RF model is overfitting?
I have a dataset with 837377 observations (51% to train, 25% to validation and 24% to test) and 19 features.
I calculated the recall score using average macro for train, validation and test and ...
0
votes
0
answers
5
views
What are the reasons that can lead to increase Micro F1-score but decrease the Macro F1-score
I have an imbalaced multi-label classification dataset
I tried these 2 models
First I used Bernoulli Naive Bayes algorithm that nativelly supports multi-label classification I got Micro F1-score of 45%...
- 517
0
votes
1
answer
66
views
Fairness metrics in the test set when wrong distribution
I have a doubt that we have been discussing for weeks with my colleagues and I wanted your opinion. I have a model for diagnosis of a disease and I want to know if it is fair. I train the model with ...
0
votes
0
answers
24
views
Handling Unbalanced dataset
I have a Tabulur dataset which is binary classification problem, where the dataset having 110000 samples of class A and class B ...
0
votes
0
answers
29
views
How to handle imbalance in input variables?
Currently working on a finance dataset which has more than 20 input variables with high imbalance.
[Apparently, the target variable is also imbalanced (for this I am currently considering to handle it ...
- 1
3
votes
2
answers
156
views
Measure distance between teeth using Machine Learning
I'm a newbie in ML and I have a problem I am stuck on.
I want to train a ML model to recognize dental diagnosis based on photos and x-rays of the patient. Specifically right now, I want to find a way ...
0
votes
0
answers
34
views
Binary Classification: My model classfies most data (95%+) as label 1
I am working with ECGs and trying to use a CNN model to perform binary classification. The goal is to classify 30s ECGs to detect a specific disease. I am using CNN and converting ECGs to images (...
- 11
1
vote
2
answers
74
views
How to decide the most suitable technique to handle Class Imbalance
For handling an imbalanced dataset, we have a variety of techniques like adjusting class weights, oversampling, undersampling, SMOTE and its different variations (RCSMOTE, GSMOTE, DBSMOTE).
My ...
- 173
1
vote
1
answer
36
views
If I'm comparing performance between two different datasets should sample and class size be uniform?
If I'm comparing performance between classification models on two different datasets should the number of samples per class, the number of classes, and features per sample be the relatively the same ...
- 13
1
vote
1
answer
22
views
4/96 imbalanced but all metrics above .95
I'm working with some severely imbalanced dataset where my 1 class represents 4% of the data in a binary classification problem.
I have about 10M rows and developed a model that outputs +.95 in ...
- 222
1
vote
0
answers
91
views
Grouped stratified train-val-test split for a multilabel dataset
So this is indeed nontrivial. I was wondering if there is a fast heuristic algorithm for performing grouped stratified dataset split on a multilabel dataset.
Stratification is usually performed to ...
- 23
0
votes
1
answer
23
views
Need my xgboost model to be more liberal with classifications
I have an xgboost model that predicts the likelihood of a sales lead to close (actually to turn into an "opportunity" which is one step before the close but that's beside the point). The ...
- 103
0
votes
0
answers
45
views
Metric for binary imbalanced classification - Case of penalized classification (class_weight = 'balanced')
I have a binary classification task with substantial class imbalance (99% negative - 1% positive). My task is to maximise the TP rate, while keeping the FN as low as possible.
I have opted to use a ...
- 11
0
votes
1
answer
39
views
Difference between class_weight and loss_weights arguments in TensorFlow/Keras
I am creating a neural network using TensorFlow (v2.9.2) for an imbalanced image dataset. While doing so, I noticed that model.compile() method has an argument <...
- 35
0
votes
0
answers
25
views
Prior probability shift vs oversampling/undersampling imbalanced datasets
I'm trying to understand what prior probability shift (label drift) in data means.
If I understand it correctly then it means that distribution of labels in training dataset differs compared to ...
- 113
0
votes
0
answers
23
views
Understanding Cost-sensitive Decision Trees Behaviour
I have a binary classification problem with imbalanced data and am attempting to use cost-sensitive learning to handle the imbalance. I have used LogisticRegression, LinearSVC, SVC and DecisionTree ...
- 367
0
votes
0
answers
21
views
Determining the information loss due to undersampling
I have an image dataset that I need to segment into directories (train, validation and test) using ImageDataGenerator in TensorFlow/Keras. The dataset is highly imbalanced:
For this I have decided to ...
- 35
0
votes
0
answers
7
views
how to find documents similar to a predefined set of documents
From big population of documents I would like to find those similar to a predefined set of documents.
All documents inside the set are similar to each other, but very few documents from the population ...
0
votes
0
answers
49
views
Early stopping on validation loss or macro-F1?
I am working on an extremely imbalanced dataset to build a classification model. The number of classes is 53 classes. I use early stopping on the validation loss to prevent the model from overfitting.
...
- 252
1
vote
1
answer
70
views
Do Sampling before or after TFIDF step?
This is a multiclass text classification problem. The dataset has a class imbalance and I'm planning to use a sampling technique before modeling.
Should the sampling be done before/after the ...
- 394
3
votes
1
answer
272
views
Why is Data with an Overrepresented Class called Imbalanced not Unbalanced?
I've seen the term Imbalanced used to described data that has an over-representation of one class.
What's the reasoning behind naming this type of data Imbalanced as opposed to Unbalanced, which seems ...
- 523
0
votes
0
answers
27
views
Classification to predict imbalanced dataset with rare event?
I'm dealing with a dataset to train a model to predict the number occurrences of events (probability(event1), probability(event2),etc). I have over 400 features. However, as shown in the table below, ...
- 101
4
votes
2
answers
665
views
Flipping the labels in a binary classification gives different model and results
I have an imbalanced dataset and I want to train a binary classifier to model the dataset.
Here was my approach which resulted into (relatively) acceptable performance:
1- I made a random split to get ...
- 43
0
votes
0
answers
13
views
Do we need to apply weights if we are using the AUC PR for imbalanced data?
If we want to apply a classification model to imbalanced data we should
Use a cost matrix or apply weights.
Use the AUC PR curve instead of the Accuracy metric to optimize my model (or the AUC ROC).
...
- 151
0
votes
0
answers
120
views
Downsampling in sklearn. Test and Train performance question
I have a class imbalanced data set, and have the following set up to handle class imbalance. I first split to test and train and only perform downsampling on the training set and then get the test ...
- 121
0
votes
0
answers
33
views
Feature Importance and Threshold Moving
Problem Type : Binary Classification
Dataset : Imbalanced
Current sklearn pipeline uses XGBoost model and involves moving threshold from 0.5 to a considerably higher value like 0.8 - 0.9.
Is it viable ...
- 1
1
vote
1
answer
51
views
Why is my model overfitting?
I am building a classification model based on some machine performance data. Unfortunately for me, it seems to over-fit no matter what I change. The dataset is quite large so I'll share the final ...
1
vote
1
answer
92
views
Why is gradient boosting better than random forest for unbalanced data?
I've searched everywhere and still couldn't figure this one out.
This post mentioned that Gradient Boosting is better than Random Forest for unbalanced data. Why is that? Is Random Forest worse ...
0
votes
0
answers
70
views
Different training score but same test score when using pipeline
I have a problem that produce different training score when using pipeline and manual.
MANUAL :
...
- 138
0
votes
0
answers
122
views
Oversampling SMOTE sampling strategy ratio
I have 36168 data with imbalanced target. 88,3% is 0 (31970 data) and 11,7% is 1 (4198 data).
I want to apply oversampling using SMOTE. Is it ideal to make it the same amount of data so the 0 & 1 ...
- 138
0
votes
0
answers
251
views
AUC OvO vs AUC OvR vs F1-Score in Multiclass Model Selection, what is better?
Given a multiclass classification task, I am looking at the best metric between AUC OvO, AUC OvR and F1-Score score to compare models.
The class distribution is the following (there is no 'class ...
0
votes
0
answers
20
views
Which model to use for multitarget classification with strong class imbalance and many categorical variables?
I have a small dataset 79 observations in 21 variables.
Almost all the variables are categorical variables in the format yes/no or 1/2/3.
I would like to predict jointly three of these variables ...
1
vote
1
answer
276
views
Plotting a no-skill model in a precision-recall curve
I am following this tutorial to apply threshold tuning using precision-recall curve for an imbalanced dataset
Within the tutorial, a no-skill model is defined as:
A no-skill model is represented by a ...
- 367
0
votes
1
answer
59
views
How can I say if I have a class imbalance issue in my data?
Assuming I have this dataset:
Label --- %Total
0 -------- 18.53%
1 -------- 8.18%
2 -------- 26.22%
3 -------- 16.46%
4 -------- 8.62%
5 -------- 9.58%
6 -------- 5.88%
7 -------- 6.53%
I could say I ...
0
votes
0
answers
19
views
Why we cannot calculate an ROC curve in cost sensitive learning?
In the Applied Predictive Modeling book, cost sensitivity learning approach, the author(s) write:
One consequence of this approach is that class probabilities cannot be
generated for the model, at ...
0
votes
0
answers
32
views
Model with no historical data
I need to develop a new credit default classification model for which there are a lot of features available but very few historical data (because it's a new activity launched by the company I work for)...
- 171
2
votes
1
answer
65
views
Why does prediction calibration on a resampling mode does not meet the expectation?
I am doing a small project to predict the write-off probability of our defaulted customers.
In the original population, the write-off rate is about 0.515. Now, for some reason I have to undersample ...
- 21
0
votes
1
answer
73
views
Classification on severe Class Imbalance high dimensional data
Dear DataScience Community, I am working on class imbalance tabular data with high-dimension inputs. The tabular data is derived from the satellite data pixels, and I have inflated the train data ...
- 101
3
votes
2
answers
168
views
Precision, recall and importance of them in the imbalance problem
I have a test dataset. The dataset is an imbalanced dataset. The total training instances for the dataset is 543 among them minority class(yes) is 75 and the majority class(No) is 468. The class of ...
- 359
2
votes
2
answers
490
views
How to calculate accuracy of an imbalanced dataset
I like to understand what is the accuracy of an imbalanced dataset.
Let's suppose we have a medical dataset and we want to predict the disease among the patients. Say, in an existing dataset 95% of ...
- 359
0
votes
1
answer
34
views
Top N accuracy for an imbalanced multiclass classification problem
I have a multi-class classification problem that is imbalanced. The task is about animal classification.
Since it's imbalanced, I am using macro-F1 metric and the current result that I have is: ...
- 252
0
votes
0
answers
69
views
The paradox of Imbalanced binary classification ¿To do something or to do nothing?
Context: Suppose we are interested in deploy a machine learning model to solve a problem of binary classification; furthermore, assume that the dataset $\mathcal{D}$ for the training of our model ...
- 123
1
vote
1
answer
201
views
scale_pos_weight effect in XGBClassifier
I can't find a satisfactory explanation about the effect of scale_pos_weight on an XGBClassifier.
It says everywhere to set it to Count of negatives / Count of ...
- 171