Questions tagged [class-imbalance]

Questions referring to classifiers or classifying problems where some of the classes in the data are under-represented.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Sampling strategies in multi-target classification

I am dealing with multi-target binary classifications (I have two targets). I need to use a sampling strategy. I have tried imblearn.pipeline but I'm getting the same error as this time when I'm ...
  • 101
1 vote
1 answer
31 views

I used SMOTE-ENN to balance my dataset and it improved the performance metrics, but how can I be sure it's not overfitting?

The models were evaluated using 10-fold cross validation. foldCount = StratifiedKFold(10, shuffle=True, random_state=1) The models in question are XGBoost. ...
  • 15
1 vote
1 answer
12 views

Class Imbalance in Dataset of Images

When dealing with an imbalanced dataset, I have been taught to oversample on only the train samples and not the entire dataset to avoid overfitting, however this was for structured text based data in ...
  • 11
0 votes
0 answers
17 views

Classification problem with a numerical variable that uses a special (high) value to indicate a qualitatively different status

I have a classification problem where I need to predict an outcome based on 20+ variables, some categorical, some numerical. One of the numerical variables is 'dlast' - which is the number of days ...
  • 101
0 votes
1 answer
14 views

Stratifed time series split with the same imbalance ratio

I am recently working on an imbalanced binary classification problem where the data is time ordered. I would like to validate my model using training/validation splits that have the same imbalance ...
1 vote
1 answer
21 views

Movement in cohorts

I am working on a user sales data which gets updated week over week. Based on the sales done in each week, the user is categorized in segment A, B or C. This means size of each segment could change ...
  • 31
0 votes
1 answer
17 views

Question about collapsing variable and oversampling minority classes

i have imbalanced data consisting of nine classes, and i am planning to collapse them into two classes. i performed stratified (proportionate) sampling between test, validation, and training sets ...
0 votes
1 answer
23 views

Can I use macro recall to check if my RF model is overfitting?

I have a dataset with 837377 observations (51% to train, 25% to validation and 24% to test) and 19 features. I calculated the recall score using average macro for train, validation and test and ...
0 votes
0 answers
5 views

What are the reasons that can lead to increase Micro F1-score but decrease the Macro F1-score

I have an imbalaced multi-label classification dataset I tried these 2 models First I used Bernoulli Naive Bayes algorithm that nativelly supports multi-label classification I got Micro F1-score of 45%...
  • 517
0 votes
1 answer
66 views

Fairness metrics in the test set when wrong distribution

I have a doubt that we have been discussing for weeks with my colleagues and I wanted your opinion. I have a model for diagnosis of a disease and I want to know if it is fair. I train the model with ...
0 votes
0 answers
24 views

Handling Unbalanced dataset

I have a Tabulur dataset which is binary classification problem, where the dataset having 110000 samples of class A and class B ...
0 votes
0 answers
29 views

How to handle imbalance in input variables?

Currently working on a finance dataset which has more than 20 input variables with high imbalance. [Apparently, the target variable is also imbalanced (for this I am currently considering to handle it ...
  • 1
3 votes
2 answers
156 views

Measure distance between teeth using Machine Learning

I'm a newbie in ML and I have a problem I am stuck on. I want to train a ML model to recognize dental diagnosis based on photos and x-rays of the patient. Specifically right now, I want to find a way ...
0 votes
0 answers
34 views

Binary Classification: My model classfies most data (95%+) as label 1

I am working with ECGs and trying to use a CNN model to perform binary classification. The goal is to classify 30s ECGs to detect a specific disease. I am using CNN and converting ECGs to images (...
  • 11
1 vote
2 answers
74 views

How to decide the most suitable technique to handle Class Imbalance

For handling an imbalanced dataset, we have a variety of techniques like adjusting class weights, oversampling, undersampling, SMOTE and its different variations (RCSMOTE, GSMOTE, DBSMOTE). My ...
  • 173
1 vote
1 answer
36 views

If I'm comparing performance between two different datasets should sample and class size be uniform?

If I'm comparing performance between classification models on two different datasets should the number of samples per class, the number of classes, and features per sample be the relatively the same ...
1 vote
1 answer
22 views

4/96 imbalanced but all metrics above .95

I'm working with some severely imbalanced dataset where my 1 class represents 4% of the data in a binary classification problem. I have about 10M rows and developed a model that outputs +.95 in ...
  • 222
1 vote
0 answers
91 views

Grouped stratified train-val-test split for a multilabel dataset

So this is indeed nontrivial. I was wondering if there is a fast heuristic algorithm for performing grouped stratified dataset split on a multilabel dataset. Stratification is usually performed to ...
0 votes
1 answer
23 views

Need my xgboost model to be more liberal with classifications

I have an xgboost model that predicts the likelihood of a sales lead to close (actually to turn into an "opportunity" which is one step before the close but that's beside the point). The ...
0 votes
0 answers
45 views

Metric for binary imbalanced classification - Case of penalized classification (class_weight = 'balanced')

I have a binary classification task with substantial class imbalance (99% negative - 1% positive). My task is to maximise the TP rate, while keeping the FN as low as possible. I have opted to use a ...
  • 11
0 votes
1 answer
39 views

Difference between class_weight and loss_weights arguments in TensorFlow/Keras

I am creating a neural network using TensorFlow (v2.9.2) for an imbalanced image dataset. While doing so, I noticed that model.compile() method has an argument <...
0 votes
0 answers
25 views

Prior probability shift vs oversampling/undersampling imbalanced datasets

I'm trying to understand what prior probability shift (label drift) in data means. If I understand it correctly then it means that distribution of labels in training dataset differs compared to ...
  • 113
0 votes
0 answers
23 views

Understanding Cost-sensitive Decision Trees Behaviour

I have a binary classification problem with imbalanced data and am attempting to use cost-sensitive learning to handle the imbalance. I have used LogisticRegression, LinearSVC, SVC and DecisionTree ...
  • 367
0 votes
0 answers
21 views

Determining the information loss due to undersampling

I have an image dataset that I need to segment into directories (train, validation and test) using ImageDataGenerator in TensorFlow/Keras. The dataset is highly imbalanced: For this I have decided to ...
0 votes
0 answers
7 views

how to find documents similar to a predefined set of documents

From big population of documents I would like to find those similar to a predefined set of documents. All documents inside the set are similar to each other, but very few documents from the population ...
0 votes
0 answers
49 views

Early stopping on validation loss or macro-F1?

I am working on an extremely imbalanced dataset to build a classification model. The number of classes is 53 classes. I use early stopping on the validation loss to prevent the model from overfitting. ...
  • 252
1 vote
1 answer
70 views

Do Sampling before or after TFIDF step?

This is a multiclass text classification problem. The dataset has a class imbalance and I'm planning to use a sampling technique before modeling. Should the sampling be done before/after the ...
3 votes
1 answer
272 views

Why is Data with an Overrepresented Class called Imbalanced not Unbalanced?

I've seen the term Imbalanced used to described data that has an over-representation of one class. What's the reasoning behind naming this type of data Imbalanced as opposed to Unbalanced, which seems ...
  • 523
0 votes
0 answers
27 views

Classification to predict imbalanced dataset with rare event?

I'm dealing with a dataset to train a model to predict the number occurrences of events (probability(event1), probability(event2),etc). I have over 400 features. However, as shown in the table below, ...
  • 101
4 votes
2 answers
665 views

Flipping the labels in a binary classification gives different model and results

I have an imbalanced dataset and I want to train a binary classifier to model the dataset. Here was my approach which resulted into (relatively) acceptable performance: 1- I made a random split to get ...
  • 43
0 votes
0 answers
13 views

Do we need to apply weights if we are using the AUC PR for imbalanced data?

If we want to apply a classification model to imbalanced data we should Use a cost matrix or apply weights. Use the AUC PR curve instead of the Accuracy metric to optimize my model (or the AUC ROC). ...
  • 151
0 votes
0 answers
120 views

Downsampling in sklearn. Test and Train performance question

I have a class imbalanced data set, and have the following set up to handle class imbalance. I first split to test and train and only perform downsampling on the training set and then get the test ...
  • 121
0 votes
0 answers
33 views

Feature Importance and Threshold Moving

Problem Type : Binary Classification Dataset : Imbalanced Current sklearn pipeline uses XGBoost model and involves moving threshold from 0.5 to a considerably higher value like 0.8 - 0.9. Is it viable ...
  • 1
1 vote
1 answer
51 views

Why is my model overfitting?

I am building a classification model based on some machine performance data. Unfortunately for me, it seems to over-fit no matter what I change. The dataset is quite large so I'll share the final ...
1 vote
1 answer
92 views

Why is gradient boosting better than random forest for unbalanced data?

I've searched everywhere and still couldn't figure this one out. This post mentioned that Gradient Boosting is better than Random Forest for unbalanced data. Why is that? Is Random Forest worse ...
0 votes
0 answers
70 views

Different training score but same test score when using pipeline

I have a problem that produce different training score when using pipeline and manual. MANUAL : ...
0 votes
0 answers
122 views

Oversampling SMOTE sampling strategy ratio

I have 36168 data with imbalanced target. 88,3% is 0 (31970 data) and 11,7% is 1 (4198 data). I want to apply oversampling using SMOTE. Is it ideal to make it the same amount of data so the 0 & 1 ...
0 votes
0 answers
251 views

AUC OvO vs AUC OvR vs F1-Score in Multiclass Model Selection, what is better?

Given a multiclass classification task, I am looking at the best metric between AUC OvO, AUC OvR and F1-Score score to compare models. The class distribution is the following (there is no 'class ...
0 votes
0 answers
20 views

Which model to use for multitarget classification with strong class imbalance and many categorical variables?

I have a small dataset 79 observations in 21 variables. Almost all the variables are categorical variables in the format yes/no or 1/2/3. I would like to predict jointly three of these variables ...
1 vote
1 answer
276 views

Plotting a no-skill model in a precision-recall curve

I am following this tutorial to apply threshold tuning using precision-recall curve for an imbalanced dataset Within the tutorial, a no-skill model is defined as: A no-skill model is represented by a ...
  • 367
0 votes
1 answer
59 views

How can I say if I have a class imbalance issue in my data?

Assuming I have this dataset: Label --- %Total 0 -------- 18.53% 1 -------- 8.18% 2 -------- 26.22% 3 -------- 16.46% 4 -------- 8.62% 5 -------- 9.58% 6 -------- 5.88% 7 -------- 6.53% I could say I ...
0 votes
0 answers
19 views

Why we cannot calculate an ROC curve in cost sensitive learning?

In the Applied Predictive Modeling book, cost sensitivity learning approach, the author(s) write: One consequence of this approach is that class probabilities cannot be generated for the model, at ...
0 votes
0 answers
32 views

Model with no historical data

I need to develop a new credit default classification model for which there are a lot of features available but very few historical data (because it's a new activity launched by the company I work for)...
  • 171
2 votes
1 answer
65 views

Why does prediction calibration on a resampling mode does not meet the expectation?

I am doing a small project to predict the write-off probability of our defaulted customers. In the original population, the write-off rate is about 0.515. Now, for some reason I have to undersample ...
0 votes
1 answer
73 views

Classification on severe Class Imbalance high dimensional data

Dear DataScience Community, I am working on class imbalance tabular data with high-dimension inputs. The tabular data is derived from the satellite data pixels, and I have inflated the train data ...
3 votes
2 answers
168 views

Precision, recall and importance of them in the imbalance problem

I have a test dataset. The dataset is an imbalanced dataset. The total training instances for the dataset is 543 among them minority class(yes) is 75 and the majority class(No) is 468. The class of ...
  • 359
2 votes
2 answers
490 views

How to calculate accuracy of an imbalanced dataset

I like to understand what is the accuracy of an imbalanced dataset. Let's suppose we have a medical dataset and we want to predict the disease among the patients. Say, in an existing dataset 95% of ...
  • 359
0 votes
1 answer
34 views

Top N accuracy for an imbalanced multiclass classification problem

I have a multi-class classification problem that is imbalanced. The task is about animal classification. Since it's imbalanced, I am using macro-F1 metric and the current result that I have is: ...
  • 252
0 votes
0 answers
69 views

The paradox of Imbalanced binary classification ¿To do something or to do nothing?

Context: Suppose we are interested in deploy a machine learning model to solve a problem of binary classification; furthermore, assume that the dataset $\mathcal{D}$ for the training of our model ...
1 vote
1 answer
201 views

scale_pos_weight effect in XGBClassifier

I can't find a satisfactory explanation about the effect of scale_pos_weight on an XGBClassifier. It says everywhere to set it to Count of negatives / Count of ...
  • 171

1
2 3 4 5
11