Questions tagged [class-imbalance]
Questions referring to classifiers or classifying problems where some of the classes in the data are under-represented.
364
questions
1
vote
0answers
14 views
How to tune an weighted voting ensemble method?
I am working on kidney cancer patients' data with 5 unbalanced labels. These codes are contained of Normalization, Oversampling on Feature Engineering part. A list of 9 ordinary Machine Learning ...
0
votes
0answers
13 views
Using accuracy metric during training for unbalanced multiclass classification
I am training a convolutional neural network and the sensitivity and precision of the minority class is what is most important to me. I am using 10-Fold cross validation, and the test fold is ...
-1
votes
1answer
20 views
Binary classification problem on imbalanced data
Literature mainly says that in general is a good idea to apply some technique to balance the two classes. For a Neural Net, what is most important here? The fact of having an imbalanced dataset (freq. ...
2
votes
1answer
36 views
When developing machine learning models, is the size of each class in the test set important?
I am thinking about the prospective application of a trained classifier in a real-world context. We know that when we do over/under-sampling to balance our dataset, we never touch the testing set as ...
1
vote
1answer
22 views
Ranking problem and imbalanced dataset
I know about the problems that imbalanced dataset will cause when we are working on classification problems. And I know the solution for that including undersampling and oversampling.
I have to work ...
0
votes
1answer
22 views
Which metric should I use for classifying an imbalace data with fewer labels for the negative class?
From reading, I understand that when we have fewer positive class labels, it is better to use precision or recall as the evaluation metric. Which metric should I use when we have fewer negative ...
1
vote
1answer
32 views
Macro and micro average for imbalanced binary classes
Micro and macro averaging are metrics for multi-class classification. However, for binary classification when data are imbalanced, it seems that micro and macro precision have different results. My ...
0
votes
0answers
16 views
how to interpret this 'lift chart'? prediction and true labels
i am trying to compare the prediction from my classifcation model and it's true label either 0 or 1.
...
4
votes
4answers
130 views
Is an $F_1$ score of 0.1 always bad?
I'm currently building a model to predict early mortgage delinquency (60+ days delinquent within 2 years of origination) for loans originating in 2018Q1. I will eventually train out-of-time (on loans ...
-1
votes
1answer
26 views
How Should I deal with my imbalanced binary target [closed]
I am trying to model my data with Python and i am having concerns about my binary target variable, because it has 90% cases falling in 0 and 10% of the cases falling in 1. I have tried upsampling my ...
0
votes
0answers
66 views
how to choose importance_type in lgbm.plot_importance
I have made a binary-classifier using lgbm.The classifier is made on unbalanced dataset.
I wanted to see the importance features of the model. There are two types of selecting importance_type -
...
-2
votes
1answer
24 views
How to achieve better accuracy of 90+ on a 3 class highly skewed dataset?
I have a 3 class dataset with very high imbalance classes:
class 1: 75000
class 2: 27000
class 3: 3000
With simple learning algorithms, accuracy is ...
0
votes
1answer
26 views
How to set a class_weight Dictionary for Random Forest?
I'm dealing with an unbalanced dataset, so I decided to use a weight dictionary for classification.
Documentation says that a weight dict must be defined as shown below:
https://imbalanced-learn.org/...
0
votes
1answer
24 views
Model accuracy: how to determine it?
I have some doubts regarding the approach to build a classifier such as Multinomial Naive Bayes or SVM. I will go through the steps to see if the approach is fine. I have not a lot of experience in ...
0
votes
2answers
51 views
What makes the validation set a good representative of the test set? [closed]
I am developing a classification model using an imbalanced dataset. I am trying to use different sampling techniques to improve the model performance.
For my baseline model, I defined an AdaBoost ...
1
vote
0answers
18 views
Image segmentation with large class imbalance leads to zero precision/recall
I have a binary semantic image classification problem where only very small parts of the images are positive, most of it is negative. In the training data I have a positive rate of around 0.023, which ...
4
votes
1answer
127 views
Why you shouldn't upsample before cross validation
I have an imbalanced dataset and I am trying different methods to address the data imbalance. I found this article that explains the correct way to cross validate when oversampling data using SMOTE ...
-1
votes
1answer
31 views
How to identify Overfitting in RandomForestClassifier?
Im building a sentiment classification model using RandomForestClassifier. I got the training accuracy of 99.65 & cross-validation( RepeatedStratifiedKFold-5 folds) accuracy of 97.29. I used f1 ...
1
vote
1answer
18 views
Test Binary Classifier on a Test-set that includes only one class
I'm working on a disease binary classification problem.
0 = healthy , 1 = not healthy
The disease is a movement disorder that appears on the patient while moving a specific movement.
I applied leave-...
0
votes
1answer
21 views
[under/over]-sampling teaches model the wrong distribution?
TLDR: Will under/oversampling during the training phase teach the model the wrong distribution and adversely affect accuracy?
Let us assume you want to train a classifier to differentiate between ...
0
votes
2answers
23 views
Creating dataset - imbalanced or balanced?
I'm trying to make an image classification model and I have 5 classes - A, B, C, D, E. The goal is to get the highest possible classification accuracy.
I have a database of images and I'm selecting ...
0
votes
0answers
22 views
SMOTE for imbalance data
I am dealing with data which has only categorical features, whose values are just 0 or 1 and it is imbalance data. So, is it good to try smote because it will try to generate new data points. but, how ...
0
votes
0answers
12 views
How does my score ranges change when I use class weight in Keras vs when I don't?
Following are images of Unweighted Score Range and Weighted Score Range
1
vote
0answers
30 views
How can I use the Brier Skill Score in cross-validation for imbalanced data?
I wish to use search for model hyper-parameters (by a grid search) using the Brier score as the scoring method (see code below):
...
1
vote
1answer
21 views
Preparing Dataset Minority Class vs Majority Class
I'm currently doing a binary classification for sentiment prediction. Currently I have the majority class (~90% of the data) as my positive class (labelled 1) and the minority class (~10% of the data) ...
0
votes
0answers
26 views
Imbalanced classification or Regression? What is the best approach to my A/B testing related problem?
The context of the problem is A/B testing of two new versions of a game. I have a structured dataset (50000 rows x 22 columns) from the game designers that represents data with respect to two versions ...
1
vote
0answers
34 views
What is the correct way to use Active Learning on a huge and unbalanced text dataset?
I am working on a huge and unlabeled text dataset. It has over 30M lines (= sentences).
We are trying to detect illegal sentences by analyzing 20 different legal issues (e.g. racism, insults, etc.). ...
2
votes
1answer
25 views
Is providing class weight to neural network enough for imbalanced binary classification?
I have a highly imbalanced binary classification problem, probably 95:5 for two classes. I don't want to perform resampling as the data is already huge and training it would just take more time. (I'm ...
1
vote
2answers
140 views
Imbalanced Dataset (Transformers): How to Decide on Class Weights?
I'm using SimpleTranformers to train and evaluate a model.
Since the dataset I am using is severely imbalanced, it is recommended that I assign weights to each ...
2
votes
3answers
68 views
How to do class balancing?
I am working with a really imbalanced dataset ($\approx$ 1% of positive cases) for a classification problem. I know that class balancing is an important step in this scenario.
I have two questions:
...
3
votes
2answers
131 views
Cost sensitive learning and class balancing
I am facing a classification problem with classes that are really imbalanced (more or less 1% of positive cases). In addition, the "cost" of a False Negative (FN) is much higher than the ...
1
vote
1answer
37 views
Selecting threshold for F1 Score
When selecting a probability threshold to maximize the F1 score prior to deploying a model (based on the precision-recall curve), should the threshold be selected based on the training or holdout ...
2
votes
1answer
99 views
SMOTE oversampling for class imbalanced dataset introduces bias in final distribution
I have a problem statement where percentage of goods (denoted by 0) is 95%, and for bads (denoted by 1) it is 5% only. One way is to do under sampling of goods so that model understands the patterns ...
1
vote
0answers
29 views
calibrating classifier probabilities for unbalanced data when class ratios are unknown
I've built a binary classification convolutional neutral network, trained on simulated data with equal numbers of simulations for each class. I've obtained good results for a validation set with equal ...
2
votes
3answers
46 views
How much of a problem is each member of a batch having the same label?
I have a batch size of 128 and a total data size of around 10 million, and I am classifying between 4 different label values.
How much of a problem is it if each batch only contains data with one ...
1
vote
1answer
18 views
Should the weights for CrossEntropyLoss be exactly the inverse of the propotions of training data?
I have a classifier network which chooses one of three classifications, and uses cross entropy loss as the loss function. If the proportions of training data are 100:10:5 for each classification, ...
0
votes
3answers
28 views
Problem of having imbalanced classes in the test set while using oversampling [closed]
I have an imbalanced dataset. My classes are 0 and 1. The number of 0 class instances is about 20 times more than the number 1 class instances. I know that I should apply oversampling after train test ...
0
votes
0answers
215 views
Image Segmentation Class weight using tensorflow keras
I remember definitely being able to pass a list to class_weight with keras (binary image segmentation specifically). For example:
...
2
votes
1answer
94 views
Using SMOTENC in a pipeline
I am trying to figure out the appropriate way to build a pipeline to train a model which includes using the SMOTENC algorithm:
Given that the N-Nearest Neighbors algorithm and Euclidian distance are ...
1
vote
2answers
51 views
Should augmentation also be performed on the validation set when the dataset is imbalanced?
I am training a CNN on images (2 classes) and I have an imbalanced dataset (1:7 ratio). I am trying to tackle this by performing offline image augmentation. Should I perform augmentation also on the ...
2
votes
1answer
47 views
Which between random forest or extra tree is best in a unbalance dataset?
I have an unbalanced dataset, with 3 classes, with 60% of class 1, 38% of class 2, and 2% of class 3.
I don't want to generate more examples of class 3, and I cannot get more examples of class 3.
The ...
1
vote
1answer
27 views
Appropriate Machine Learning algorithm for modeling clustered time-varying binary outcome
I'll just dive right in. I have a decent-size (100K observations) dataset of time-varying continuous and categorical predictors. Categorical predictors, actually, usually do not change, however, ...
0
votes
1answer
33 views
XG Boost result interpretation for unbalanced datasets (Accuracy & AUCROC)
My dataset is of shape – 5621*8 (binary classification)
Label/target : Success (4324, 77 %) & Not success (1297, 23 %)
(success and Not success were been ...
1
vote
1answer
54 views
Cross validation schema for imbalanced dataset
Based on a previous post, I understand the need to ensure that the validation folds during the CV process have the same imbalanced distribution as the original dataset when training a binary ...
0
votes
0answers
18 views
How do I build a model to improve CTR on campaign?
I am trying to build a propensity model for a client to increase the CTR.
Client has the list of people who clicked in the previous campaigns but doesn't have the data on the list of people who didn't ...
2
votes
2answers
177 views
For imbalanced classification, should the validation dataset be balanced?
I am building a binary classification model for imbalanced data (e.g., 90% Pos class vs 10% Neg Class).
I already balanced my training dataset to reflect a a 50/50 class split, while my holdout (...
0
votes
1answer
48 views
Semi-supervised anomaly detection
I am currently exploring anomaly detection methods for my work and, basically I have gone through Local Oulier Factor and Isolation Forests, both unsupervised methods.
Now, the thing is, there might ...
1
vote
2answers
59 views
Main options on how to deal with imbalanced data
As far as I can tell, broadly speaking, there are three ways of dealing with binary imbalanced datasets:
Option 1:
Create k-fold Cross-Validation samples randomly (or even better create k-fold ...
1
vote
0answers
43 views
Orange ROC analysis widget [closed]
ROC analysis widget has an option to change prior target class probability. I'd like to know, when and how it should be used. Playing with it changes the slope of iso-performance line. Each classifier ...
2
votes
2answers
53 views
Data Imbalance in Regression Tasks [closed]
Having read a lot about class imbalance in classification tasks, I'd like to know what is the methodology for data imbalance in regression tasks.
Particularly,
- What is the procedure to check for ...
