Questions tagged [class-imbalance]

Questions referring to classifiers or classifying problems where some of the classes in the data are under-represented.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Understanding Cost-sensitive Decision Trees Behaviour

I have a binary classification problem with imbalanced data and am attempting to use cost-sensitive learning to handle the imbalance. I have used LogisticRegression, LinearSVC, SVC and DecisionTree ...
  • 355
0 votes
0 answers
14 views

Determining the information loss due to undersampling

I have an image dataset that I need to segment into directories (train, validation and test) using ImageDataGenerator in TensorFlow/Keras. The dataset is highly imbalanced: For this I have decided to ...
0 votes
0 answers
4 views

how to find documents similar to a predefined set of documents

From big population of documents I would like to find those similar to a predefined set of documents. All documents inside the set are similar to each other, but very few documents from the population ...
0 votes
0 answers
12 views

Early stopping on validation loss or macro-F1?

I am working on an extremely imbalanced dataset to build a classification model. The number of classes is 53 classes. I use early stopping on the validation loss to prevent the model from overfitting. ...
  • 252
1 vote
1 answer
15 views

Do Sampling before or after TFIDF step?

This is a multiclass text classification problem. The dataset has a class imbalance and I'm planning to use a sampling technique before modeling. Should the sampling be done before/after the ...
3 votes
1 answer
252 views

Why is Data with an Overrepresented Class called Imbalanced not Unbalanced?

I've seen the term Imbalanced used to described data that has an over-representation of one class. What's the reasoning behind naming this type of data Imbalanced as opposed to Unbalanced, which seems ...
  • 175
0 votes
0 answers
22 views

Classification to predict imbalanced dataset with rare event?

I'm dealing with a dataset to train a model to predict the number occurrences of events (probability(event1), probability(event2),etc). I have over 400 features. However, as shown in the table below, ...
  • 101
4 votes
2 answers
494 views

Flipping the labels in a binary classification gives different model and results

I have an imbalanced dataset and I want to train a binary classifier to model the dataset. Here was my approach which resulted into (relatively) acceptable performance: 1- I made a random split to get ...
  • 43
0 votes
0 answers
12 views

Do we need to apply weights if we are using the AUC PR for imbalanced data?

If we want to apply a classification model to imbalanced data we should Use a cost matrix or apply weights. Use the AUC PR curve instead of the Accuracy metric to optimize my model (or the AUC ROC). ...
  • 151
0 votes
0 answers
24 views

Downsampling in sklearn. Test and Train performance question

I have a class imbalanced data set, and have the following set up to handle class imbalance. I first split to test and train and only perform downsampling on the training set and then get the test ...
  • 121
0 votes
0 answers
22 views

Feature Importance and Threshold Moving

Problem Type : Binary Classification Dataset : Imbalanced Current sklearn pipeline uses XGBoost model and involves moving threshold from 0.5 to a considerably higher value like 0.8 - 0.9. Is it viable ...
  • 1
1 vote
1 answer
47 views

Why is my model overfitting?

I am building a classification model based on some machine performance data. Unfortunately for me, it seems to over-fit no matter what I change. The dataset is quite large so I'll share the final ...
1 vote
1 answer
40 views

Why is gradient boosting better than random forest for unbalanced data?

I've searched everywhere and still couldn't figure this one out. This post mentioned that Gradient Boosting is better than Random Forest for unbalanced data. Why is that? Is Random Forest worse ...
0 votes
0 answers
37 views

Different training score but same test score when using pipeline

I have a problem that produce different training score when using pipeline and manual. MANUAL : ...
0 votes
0 answers
36 views

Oversampling SMOTE sampling strategy ratio

I have 36168 data with imbalanced target. 88,3% is 0 (31970 data) and 11,7% is 1 (4198 data). I want to apply oversampling using SMOTE. Is it ideal to make it the same amount of data so the 0 & 1 ...
0 votes
0 answers
78 views

AUC OvO vs AUC OvR vs F1-Score in Multiclass Model Selection, what is better?

Given a multiclass classification task, I am looking at the best metric between AUC OvO, AUC OvR and F1-Score score to compare models. The class distribution is the following (there is no 'class ...
0 votes
0 answers
15 views

Which model to use for multitarget classification with strong class imbalance and many categorical variables?

I have a small dataset 79 observations in 21 variables. Almost all the variables are categorical variables in the format yes/no or 1/2/3. I would like to predict jointly three of these variables ...
0 votes
1 answer
92 views

Plotting a no-skill model in a precision-recall curve

I am following this tutorial to apply threshold tuning using precision-recall curve for an imbalanced dataset Within the tutorial, a no-skill model is defined as: A no-skill model is represented by a ...
  • 355
0 votes
1 answer
51 views

How can I say if I have a class imbalance issue in my data?

Assuming I have this dataset: Label --- %Total 0 -------- 18.53% 1 -------- 8.18% 2 -------- 26.22% 3 -------- 16.46% 4 -------- 8.62% 5 -------- 9.58% 6 -------- 5.88% 7 -------- 6.53% I could say I ...
0 votes
0 answers
14 views

Why we cannot calculate an ROC curve in cost sensitive learning?

In the Applied Predictive Modeling book, cost sensitivity learning approach, the author(s) write: One consequence of this approach is that class probabilities cannot be generated for the model, at ...
0 votes
0 answers
29 views

Model with no historical data

I need to develop a new credit default classification model for which there are a lot of features available but very few historical data (because it's a new activity launched by the company I work for)...
  • 161
2 votes
1 answer
58 views

Why does prediction calibration on a resampling mode does not meet the expectation?

I am doing a small project to predict the write-off probability of our defaulted customers. In the original population, the write-off rate is about 0.515. Now, for some reason I have to undersample ...
0 votes
1 answer
60 views

Classification on severe Class Imbalance high dimensional data

Dear DataScience Community, I am working on class imbalance tabular data with high-dimension inputs. The tabular data is derived from the satellite data pixels, and I have inflated the train data ...
3 votes
2 answers
103 views

Precision, recall and importance of them in the imbalance problem

I have a test dataset. The dataset is an imbalanced dataset. The total training instances for the dataset is 543 among them minority class(yes) is 75 and the majority class(No) is 468. The class of ...
  • 359
2 votes
2 answers
333 views

How to calculate accuracy of an imbalanced dataset

I like to understand what is the accuracy of an imbalanced dataset. Let's suppose we have a medical dataset and we want to predict the disease among the patients. Say, in an existing dataset 95% of ...
  • 359
0 votes
1 answer
30 views

Top N accuracy for an imbalanced multiclass classification problem

I have a multi-class classification problem that is imbalanced. The task is about animal classification. Since it's imbalanced, I am using macro-F1 metric and the current result that I have is: ...
  • 252
0 votes
0 answers
41 views

The paradox of Imbalanced binary classification ¿To do something or to do nothing?

Context: Suppose we are interested in deploy a machine learning model to solve a problem of binary classification; furthermore, assume that the dataset $\mathcal{D}$ for the training of our model ...
1 vote
1 answer
27 views

scale_pos_weight effect in XGBClassifier

I can't find a satisfactory explanation about the effect of scale_pos_weight on an XGBClassifier. It says everywhere to set it to Count of negatives / Count of ...
  • 161
0 votes
1 answer
19 views

Do I need to use AUPRC for reporting classification results on an imbalanced dataset when the model was trained using upsampling and CV

I am working on a binary classification problem which dataset has about 5% of positive class samples. I split the dataset, 70% for training and 30% for testing. I used the test data only once for ...
  • 1
0 votes
0 answers
31 views

the alpha balance parameter in focal loss

I want to use a weighted focal loss for my imbalanced object detection problem. let's say I have 3 classes, with 10000, 1000, and 100 examples each. The focal loss function looks like this: \begin{...
0 votes
1 answer
41 views

How to effectively evaluate a model with highly imbalanced and limited dataset

Most data imbalance questions on this stack have been asking How to learn a better model, but I tend to think one other problem is How do we define "better" (i.e. fairly evaluate the learned ...
0 votes
0 answers
7 views

Lift - Class ratio as actual randomness-measure

Context The Lift should show how a machine learning model performs better than randomness. Thus, a curve representing the ratio between the predicted class of a ...
  • 111
0 votes
0 answers
20 views

How to choose a machine learning sampling method?

I have a multiclassification problem with a training dataset of 3 groups (50 samples:150 samples: 100 samples). I have tried comparing models running SMOTE oversampling and class weighting (using ...
  • 1
0 votes
0 answers
45 views

Data Imbalance in Contextual Bandit with Thompson Sampling

I'm working with the Online Logistic Regression Algorithm (Algorithm 3) of Chapelle and Li in their paper, "An Empirical Evaluation of Thompson Sampling" (https://papers.nips.cc/paper/2011/...
  • 1
0 votes
0 answers
116 views

Using class weights with training on imbalanced dataset gives worse result w.r.t logloss than without weights

I am trying to make a model for usual binary classification that is able to predict probabilities of classes. I have not very big dataset of 10k objects where classes are imbalanced as 80:20 and tried ...
3 votes
0 answers
92 views

Zero-shot learning for tabular data?

Can anyone point me to methods for zero-shot learning on tabular data? There is some very cool work being done for zero-shot learning on images and text, but I'm struggling to find work being done to ...
0 votes
0 answers
20 views

Sampling Highly Imbalanced Large Dataset

I am working on a model which will run monthly on 8M users. I've snapshot-wise data in training set, eg: Jan, 21 Snapshot : 8M Total : 233 Positives Rest Negative Feb, 21 Snapshot : 8M Total : 599 ...
2 votes
1 answer
258 views

class weighted classification

I am working on my multi-class classification project and I have a question: I have three classes in proportion: 50%, 47% and 3%. I decided to use ...
  • 41
0 votes
1 answer
120 views

ROC-AUC Imbalanced Data Score Interpretation

I have a binary response variable (label) 𝐵 in a dataset with around 50,000 observations. The training set is somewhat imbalanced with, 𝐵𝑖=1 making up about 33% of the observation's and 𝐵𝑖=0 ...
0 votes
0 answers
173 views

under sample to get specific number of samples per class using tomek links of imblearn

I have a dataset with classes in my target column distributed like shown below. ...
1 vote
0 answers
73 views

Can data augmentation techniques be misleading?

In an attempt to handle imbalance in data, especially in the case of extremely imbalanced data, can the various data augmentation techniques create some bias?
  • 23
3 votes
1 answer
111 views

Area Under the Precision Recall Curve

I have got the following Precision Recall Curve for a classifier I built using AutoML. Most of the Precisio-Recall curves tend to start from (0, 1) go towards (1,0). But mine is the opposite. But I ...
  • 151
0 votes
0 answers
38 views

Complex balanced dataloading from multiple imbalanced datasets?

The Setting Let's suppose that I have an imbalanced dataset. For training purposes, I want to implement a dataloading scheme that samples from this dataset in a more balanced way. I want to leverage ...
0 votes
0 answers
21 views

Can you get a very good AUC-ROC score despite predicting all rows to have the same probability?

On the test set of a binary classification problem, the p25, p50 and p75 of the predictions are very close to each other (e.g. 0.123). Is it possible that my model can achieve a high AUC-ROC (e.g. 0....
1 vote
1 answer
24 views

Does synthetic data be over sampled as well?

I'm building a binary text classifier, the ratio between the positives and negatives is 1:100 (100 / 10000). By using back translation as an augmentation, I was able to get 400 more positives. Then I ...
0 votes
0 answers
28 views

Oversampling in imbalanced classification producing perfect classification

I have a 40 to 1 imbalance for my binary classification problem. I proceeded to solve the imbalance by oversampling and I generated synthetic samples to oversample my minority class. After having ...
1 vote
1 answer
63 views

How to define minority/majority class in a multi-classification task

I am studying classification in imbalanced datasets and I am learning under/over sampling strategies as a way to address the issue. While the literature agrees one needs to oversample 'minority' ...
  • 111
0 votes
1 answer
77 views

Imbalanced data: understanding example from Bishop PRML book?

I'm trying to understand the 3-step procedure to compensate for the effects of imbalanced data described in Section 1.5.4 - pg 45 of Bishop's PRML book. Please refer to the following excerpt from the ...
  • 3
1 vote
1 answer
103 views

Class imbalance: Will transforming multi-label (aka multi-task) to multi-class problem help?

I noticed this and this questions, but my problem is more about class imbalance. So now I have, say, 1000 targets and some input samples (with some feature vectors). Each input sample can have label ...
1 vote
1 answer
23 views

Will a classifier trained on undersampled data make accurate predictions on new imbalanced data?

I have a dataset with about 200,000 entries. The target variable is binary, and only 4,000 instances belong to the class of interest. I would like to undersample the majority class so that we have a ...
  • 85

1
2 3 4 5
11