Questions tagged [class-imbalance]
Questions referring to classifiers or classifying problems where some of the classes in the data are under-represented.
496
questions
0
votes
0
answers
24
views
Using class weights with training on imbalanced dataset gives worse result w.r.t logloss than without weights
I am trying to make a model for usual binary classification that is able to predict probabilities of classes. I have not very big dataset of 10k objects where classes are imbalanced as 80:20 and tried ...
1
vote
0
answers
21
views
Zero-shot learning for tabular data?
Can anyone point me to methods for zero-shot learning on tabular data? There is some very cool work being done for zero-shot learning on images and text, but I'm struggling to find work being done to ...
- 43
0
votes
0
answers
20
views
Sampling Highly Imbalanced Large Dataset
I am working on a model which will run monthly on 8M users. I've snapshot-wise data in training set, eg:
Jan, 21 Snapshot : 8M Total : 233 Positives Rest Negative
Feb, 21 Snapshot : 8M Total : 599 ...
2
votes
1
answer
147
views
class weighted classification
I am working on my multi-class classification project and I have a question:
I have three classes in proportion: 50%, 47% and 3%. I decided to use ...
- 41
0
votes
1
answer
21
views
ROC-AUC Imbalanced Data Score Interpretation
I have a binary response variable (label) 𝐵 in a dataset with around 50,000 observations.
The training set is somewhat imbalanced with, 𝐵𝑖=1 making up about 33% of the observation's and 𝐵𝑖=0 ...
- 21
0
votes
0
answers
26
views
under sample to get specific number of samples per class using tomek links of imblearn
I have a dataset with classes in my target column distributed like shown below.
...
1
vote
0
answers
37
views
Can data augmentation techniques be misleading?
In an attempt to handle imbalance in data, especially in the case of extremely imbalanced data, can the various data augmentation techniques create some bias?
- 11
3
votes
1
answer
45
views
Area Under the Precision Recall Curve
I have got the following Precision Recall Curve for a classifier I built using AutoML. Most of the Precisio-Recall curves tend to start from (0, 1) go towards (1,0). But mine is the opposite. But I ...
- 151
0
votes
0
answers
34
views
Complex balanced dataloading from multiple imbalanced datasets?
The Setting
Let's suppose that I have an imbalanced dataset.
For training purposes, I want to implement a dataloading scheme that samples from this dataset in a more balanced way.
I want to leverage ...
- 317
0
votes
0
answers
18
views
Can you get a very good AUC-ROC score despite predicting all rows to have the same probability?
On the test set of a binary classification problem, the p25, p50 and p75 of the predictions are very close to each other (e.g. 0.123).
Is it possible that my model can achieve a high AUC-ROC (e.g. 0....
- 1
1
vote
1
answer
21
views
Does synthetic data be over sampled as well?
I'm building a binary text classifier, the ratio between the positives and negatives is 1:100 (100 / 10000).
By using back translation as an augmentation, I was able to get 400 more positives.
Then I ...
0
votes
0
answers
22
views
Oversampling in imbalanced classification producing perfect classification
I have a 40 to 1 imbalance for my binary classification problem. I proceeded to solve the imbalance by oversampling and I generated synthetic samples to oversample my minority class. After having ...
1
vote
1
answer
27
views
How to define minority/majority class in a multi-classification task
I am studying classification in imbalanced datasets and I am learning under/over sampling strategies as a way to address the issue. While the literature agrees one needs to oversample 'minority' ...
- 111
0
votes
1
answer
64
views
Imbalanced data: understanding example from Bishop PRML book?
I'm trying to understand the 3-step procedure to compensate for the effects of imbalanced data described in Section 1.5.4 - pg 45 of Bishop's PRML book. Please refer to the following excerpt from the ...
- 3
1
vote
1
answer
40
views
Class imbalance: Will transforming multi-label (aka multi-task) to multi-class problem help?
I noticed this and this questions, but my problem is more about class imbalance. So now I have, say, 1000 targets and some input samples (with some feature vectors). Each input sample can have label ...
- 13
1
vote
1
answer
21
views
Will a classifier trained on undersampled data make accurate predictions on new imbalanced data?
I have a dataset with about 200,000 entries. The target variable is binary, and only 4,000 instances belong to the class of interest.
I would like to undersample the majority class so that we have a ...
- 85
1
vote
1
answer
18
views
Remedie for a stubborn recall result?
I was working on a project connected to predicting default on credit loan with 0-1 loss. The recall is a crucial measure that should be maximized in this case, while monitoring precision for sanity of ...
0
votes
1
answer
379
views
ValueError: `class_weight` is only supported for Models with a single output
I'm getting the below error while using class weights in the model.fit in tensorflow version 2.7.0
...
- 1
0
votes
1
answer
28
views
Give more weight to features based on distribution plot
I have a task to predict a binary variable purchase, their dataset is strongly imbalanced (10:100) and the models I have tried so far (mostly ensemble) fail. In ...
- 101
1
vote
0
answers
18
views
Interrupted Time Series with Unevenly Distributed Samples
I'm working on causal inference using Interrupted Time Series Design.
I have multiple samples per day and am selecting my analysis bandwidth based on pre-treatment RMSE on leave-on-out cross ...
1
vote
2
answers
77
views
Evaluation Metric for Imbalanced and Ordinal Classification
I'm looking for an ML evaluation metric that would work well with imbalanced and ordinal multiclass datasets:
Imagine you want to predict the severity of a disease that has 4 grades of severity where ...
- 111
1
vote
2
answers
93
views
Why does class_weight usually outperform SMOTE?
I'm trying to figure out what exactly class_weight from sklearn does.
When working with imbalanced datasets, I'm always using ...
- 161
0
votes
2
answers
19
views
Rough ideas of expected performance boost from over-sampling techniques?
I am trying to train a classifier for a multi class classification task. However, the dataset is very imbalanced. About half of the around 160 unique labels are such that there are only 10 or less ...
- 103
2
votes
1
answer
198
views
Imbalanced data set with Sample weighting - How to interpret the performance metrics?
Consider a binary classification scenario whereby the True class (5%) is severely outbalanced to the False class (95%). My data set contains numeric data. I am using SKLearn and trying some different ...
- 129
0
votes
2
answers
46
views
How to tell if downsampling helped model performance
Fitting a logistic classifier to imbalanced data. My target variable is 5% 1, 95% 0. As such, I think it's probably best to use the PR-AUC to evaluate the model rather than ROC-AUC. I get a PR-AUC of ...
0
votes
0
answers
23
views
How can I improve calibration curves?
I am training a binary xgboost classifer with an imbalance of : 85% = 0 class and 14 % = class 1.
This was achieved after i took a random sample fromaround 11m to 1M.
When i calibrate i get the ...
- 466
0
votes
1
answer
107
views
Improving precision and recall for imbalanced large data set
I have a data set of 1 million points and 30 features. The output variable has multiple classes (1 to $n$) but the problem I'm interested in is only concerned whether the output belongs to class 1 or ...
- 101
6
votes
1
answer
299
views
Random Forest significantly outperforms XGBoost - problem or possible?
I have dataset of around 180k observations of 13 variables (mix of numerical and categorical features). It is binary classification problem, but classes are imbalanced (25:1 for negative ones). I ...
- 63
0
votes
1
answer
56
views
Improving Performance of Machine Learning for A Small Imbalanced Dataset
I am a researcher in Machine Learning. In my project, I have been applying ML to a small imbalanced data consisting of 8 features and 297 instances with 44 positive instances and 253 negative ones. ...
- 66
0
votes
1
answer
24
views
Explaining the logic behind the pipe_line method for cross-validation of imbalance datasets
Reading the following article: https://kiwidamien.github.io/how-to-do-cross-validation-when-upsampling-data.html
There is an explanation of how to use ...
- 101
1
vote
0
answers
20
views
Handling imbalanced Feature (X) not lavbel (Y) in machine learning
I am very new to this field and have done a decent amount of research on this, but every time, I stumble upon handling the imbalanced label by using f1 score, recall, precision as metrics, and using ...
- 111
4
votes
2
answers
239
views
Why SMOTE is not used in prize-winning Kaggle solutions?
Synthetic Minority Over-sampling Technique SMOTE, is a well known method to tackle imbalanced datasets. There are many papers with a lot of citations out-there claiming that it is used to boost ...
- 5,436
1
vote
0
answers
13
views
Feature Scaling + Selection when target is imbalanced
If my target is imbalanced, when should I do target balancing in preparation for modeling?
Before feature scaling and selection? After feature scaling and selection?
If I am doing backward elimination,...
- 11
0
votes
0
answers
181
views
Steps for balancing data using SMOTE
Right now I'm doing sentiment analysis (classification) with TF-IDF and SVM linear. My data is not balanced and I want to make data balance using SMOTE from ...
- 101
1
vote
2
answers
32
views
Handling missing data for the majority class
I'm working with an unbalanced (10:1) dataset for classification. I also have a bunch of missing data on certain columns. If I discard them all, I still have a 5:1 ratio, so I guess I can afford to ...
0
votes
0
answers
454
views
Applying SMOTE on time series data
I have a dataset that consist of student grades and it's based on a time series. I used LSTM to predict the student future grade. Can I apply SMOTE on this dataset to ensure that the model will not be ...
- 101
2
votes
1
answer
44
views
How do you train a semantic segmentation model to optimize for IoU rather than accuracy?
I am currently building a U-NET semantic segmentation model on Tensorflow Keras to classify pixels as belonging to or not belonging to a class.
For this problem, I've isolated the masks for only one ...
3
votes
1
answer
599
views
How F1 score is good with unbalanced dataset
I have read around on this site that it's recommended to use F1 score if the dataset is imbalanced and if you want to seek a balance between recall and precession. Could you please explain how F1 can ...
- 179
2
votes
1
answer
34
views
Some questions about supervised learning, model evaluation and preprocessing [closed]
I've been trying to employ some basic techniques of supervised learning on a dataset that I have and I have several questions about the overall procedure (i.e. data preprocessing, model evaluation etc)...
- 123
1
vote
1
answer
97
views
How to deal with imbalanced data with train and test data
I am working with the Loan problem whether Loan Status: Defaulter or Non-Defaulters.In this problem, my classes are unbalanced 90% of classes are Defaulter, and 10% of them Non-Defaulter. Then I tried ...
- 11
1
vote
1
answer
42
views
How to split unbalanced data for supervised learning?
Suppose I have data I want to use for supervised learning, but there is a pretty bad target/class/labels imbalance. Should I:
Limit the size of the training set to make sure there is a flat target/...
- 133
0
votes
0
answers
43
views
Imbalanced NLP text classification
I'm trying to solve a multi-class text classification task with 3 classes. I have an initial pretty balanced but small dataset. When I start to mine additional data I can't always find a lot of new ...
- 157
1
vote
0
answers
14
views
What are some strategies to deal with label sparsity when training a protein function prediction model?
The protein function prediction task requires you to take a sequence of amino acids (think words in a sentence, but if there are only 20 words), and output the functions that protein can take. There ...
- 111
4
votes
2
answers
1k
views
Imbalanced Dataset: Train/test split before and after SMOTE
This question is similar but different from my previous one. I have a binary classification task related to customer churn for a bank. The dataset contains 10,000 instances and 11 features. The target ...
- 65
0
votes
1
answer
381
views
roc_auc_score from sk-learn gives error when test label vector with classes has only a subset of the whole set
I have an imbalanced dataset.
Does it make sense to compute the roc-auc for the classifier I created in a holdout set?
Here's very artificial MWE:
...
1
vote
0
answers
51
views
Should I use Pad Sequence when using Word Vectors?
I have an unbalanced text data set. I want to use word vectors to embed words. When I use pad sequence? Before or after the word vector? I tried it, after the word vector I used pad sequence but my ...
- 13
1
vote
1
answer
94
views
Imbalanced classification task – Discrepancy between learning curves and test set evaluation
I have a binary classification task related to customer churn for a bank. The dataset contains 10,000 instances and 11 features. The target variable is imbalanced (80% remained as customers (0), 20% ...
- 65
0
votes
0
answers
22
views
Class Weight in sklearn DecisionTreeClassifier impact during prediction
I understand that class weights are used during splitting to weigh whatever metric in the children of the split. However I cannot find anywhere whether class weights also impact prediction or are ...
- 103
0
votes
0
answers
33
views
Train Test Split for Imbalance Data set for credit card transaction data set
I am currently working on a credit card transaction datasets for fraud detection, and I am unsure how to go about splitting the data.
Transactions are time related data, do I split them like how you ...
1
vote
1
answer
32
views
Does class weighting encourage overfitting when the true class distribution is imbalanced?
I am working on a classification problem in which ~90% of samples come from class 1 while ~10% of samples come from class 2. I have been using various techniques to combat the class imbalance while ...
- 43