Questions tagged [class-imbalance]
Questions referring to classifiers or classifying problems where some of the classes in the data are under-represented.
394
questions
0
votes
1answer
9 views
Hypertune xgboost to dealing with imbalanced dataset
My training data has extremely class imbalanced {0:872525,1:3335} with 100 features. I use xgboost to build classification model with bayessian optimisation to hypertune the model in range {learning ...
5
votes
3answers
143 views
Metric for label imbalance
I'm looking for a metric that can be used to quantify how imbalanced the labels are in a dataset.
I'm not looking for a strategy to solve the imbalance problem, I just want to present how imbalanced ...
0
votes
1answer
33 views
Any pythonic way to auto determine imbalance class problem, specially in multiclass scenario?
A data is imbalanced if a target class proportions are unequal and typically, heavily biased. But, what is the exact measurement of this heavy bias?
Before applying imbalance techniques (SMOTE, ADASYN,...
0
votes
1answer
34 views
Handling Imbalanced Datasets
I work in the medical domain, so class imbalance is the rule and not the exception. While I know Python has packages for class imbalance, I don't see an option in Orange for e.g. a SMOTE widget. I ...
0
votes
1answer
20 views
Class imbalance and statistics
I have a dataset with 5000 observations with class 0 and 300 observations with class 1. I would like to run some statistical analysis, for example on the average length of strings, the number of words,...
0
votes
0answers
16 views
Cost Function Binary Classification
I have imbalance dataset for binary classification problem. I want to create a custom cost function that takes into account not only the actual class and probability, but another variable "...
0
votes
0answers
26 views
Confusion regarding accuracy and individual class performance
Consider a three-class classification problem where avg_cm1 and avg_cm2 are two average confusion matrices across 3 folds from ...
2
votes
2answers
66 views
Undersampling for credit card fraud detection before or after Train/Test Split
I have a credit card dataset with 98% transactions are Non-Fraud and 2% are fraud.
I have been trying to undersample the majotrity class before train and test split and get very good recall and ...
3
votes
1answer
171 views
Features selection in imbalanced dataset
I have some doubts regarding an analysis. I have a dataset with class imbalance. I am trying to investigate some information from that data, e.g., how many urls contain http or https protocols.
My ...
1
vote
1answer
32 views
What does the oob decision function mean in random forest, how get class predictions from it, and calculating oob for unbalanced samples
I am interested in finding the OOB score for random forest using sklearn, when it is used for a binary classification task, and there are unbalanced samples. What does the oob decision function mean ...
1
vote
1answer
14 views
Which metric to use for evaluating a rating system
I have a system which gives a star rating of the quality of work(on scale of 1-5, 1 being extremely poor and 5 being exceptionally good).
An expert labelled a test set with their ratings of quality of ...
4
votes
1answer
93 views
Why removing rows with NA values from the majority class improves model performance
I have an imbalanced dataset like so:
df['y'].value_counts(normalize=True) * 100
...
1
vote
1answer
25 views
Test set larger than train set [closed]
There is a two class dataset with 1121 values in total, having 230 from same class and 891 from the other class. The training set is choosen as 230+230=460 from both classes and the test set as the ...
2
votes
1answer
28 views
What does IBA mean in imblearn classification report?
imblearn is a python library for handling imbalanced data. A code for generating classification report is given below.
...
0
votes
0answers
9 views
Modelling probability of switching of label from 0 to 1 in next time period for multivariate time series data
I have a data set consisting of unbalanced panel data, i.e. longitudinal multivariate measurements on multiple individuals. I want to estimate a probability that the individual will become 1 in next ...
0
votes
1answer
40 views
In which situation should we consider a dataset as imbalanced?
I'm facing a problem about making a classification on a dataset. The target variable is binary (with 2 classes, 0 and 1). I have 8,161 samples in the training dataset. And for each class, I have:
...
1
vote
1answer
30 views
Strong overfitting accompanying strong class imbalance
I'm training an xgboost binary classification model. The data I have is around 600k and positive is only 0.1% of it. I tried to use all overfitting prevention techniques xgboost has to offer (tune eta,...
0
votes
0answers
20 views
How can I fix my classifier only predicting two classes, and do my metrics show that it is overfitting?
I have a relatively simple 16 feature neural network attempting to predict the outcome of a sports event as win, loss, draw, however regardless of the number of layers, or the number of nodes in said ...
1
vote
0answers
33 views
How do I handle class imbalance for text data when using pretrained models like BERT?
I have a skewed dataset consisting of samples of the form:
Category 1 10000
Category 2 2000
Category 3 400
Category 4 300
Category 5 100
The dataset ...
1
vote
0answers
20 views
Follow up question regarding Upsampling for Imbalanced Data and the use of ADASYN instead of SMOTE
I have a follow-up question regarding this topic.
I have been working on a project predicting success(1) or failure(0) for organizations by using the Decision Tree and Random Forest algorithms.
My ...
1
vote
1answer
27 views
Is it advisable to merge similar datasets to improve model accuracy?
I'm trying to build a classifier that would help me classify whether a statement collected from Reddit is bullish, bearish or neutral.
To this end, I have hand-labelled a fairly small dataset of 2500 ...
2
votes
1answer
52 views
How to split up my dataset in a train and testset, in order to prevent data leakage?
I realize that this could be considered a duplicate of this question, Is using samples from the same person in both trainset and testset considers being a data leakage?, where it is stated that "...
2
votes
1answer
43 views
Calibrating probability thresholds for multiclass classification
I have built a network for the classification of three classes. The network consists of a CNN followed by two fully-connected layers. The CNN consists of convolutional layers, followed by batch ...
0
votes
1answer
17 views
Many questions training unbalanced and duplicated data
I'm a DS student. I have like 30.000 of bank statements, all labeled with a specific category(cat1, cat2, ...). With that data I'm trying to train a classification model but I found several problems:
...
2
votes
2answers
54 views
Is the PR AUC invariant under label flip?
The ROC-AUC curve is invariant under a flip of the labels. I don't know if its a famous result so I will give the proof below. My question is if the PR-AUC curve also has this property. I have not ...
-1
votes
1answer
22 views
using average precision as metric for imbalanced problem (learning curve example) [closed]
I have an imbalanced problem (2% target class) and therefore need an appropriate metric - so I chose average_precision.
My code:
...
0
votes
0answers
29 views
Overfitting in imbalanced dataset
I am working on a dataset related to an insurance company and the objective is to predict if the insurance buyer will claim their travel insurance or not.
Training data: https://raw.githubusercontent....
2
votes
1answer
58 views
Cross validation for unbalanced dataset using Orange data mining tool
I am using the Orange data mining tool to build and analyze models (decision tree, ANN, ...) predicting customer churn. As this is an imbalanced class problem (10% churn, 90% not churn), I need to ...
0
votes
0answers
10 views
Classification model performance - metrics for getting number in each class correct?
I'm fairly new to predictive modelling, so apologies if this is a stupid question.
I am working on a classification problem (predicting if customers commit fraud or not), and have been comparing a few ...
0
votes
2answers
34 views
Training binary classifier on only one data point ( Theoritical question)
Say, I'm training a binary classifier to classify Dog vs Cat. Now, say I train my model only on one imagee ( cat). Now I mirror this cat image that I used to train my model. Now on the mirror image I ...
0
votes
0answers
15 views
Data leakage when setting class_weight to tackle imbalanced time series data?
I'm using a random forest classifier from sklearn to predict whether a stock's return for the next period is greater than a certain threshold (say -2%), so negative is 0 and positive is 1, a binary ...
1
vote
1answer
33 views
Labels as features in anomaly detection
I have a dataset born to solve a classification problem. Due to the imbalances of the Y, i choose to move to an anomaly detection task. Should I use the Y i have inside the anomaly detection model as ...
0
votes
0answers
16 views
How many instances should be synthesized for each class when using over-sampling techniques?
As for an imbalanced multi-class dataset, how many instances should be synthesized for each class if we use over-sampling techniques such as SMOTE?
For example, there is 4 class including 'A', 'B', 'C'...
2
votes
1answer
42 views
How to tune an weighted voting ensemble method?
I am working on kidney cancer patients' data with 5 unbalanced labels. These codes are contained of Normalization, Oversampling on Feature Engineering part. A list of 9 ordinary Machine Learning ...
0
votes
0answers
17 views
Using accuracy metric during training for unbalanced multiclass classification
I am training a convolutional neural network and the sensitivity and precision of the minority class is what is most important to me. I am using 10-Fold cross validation, and the test fold is ...
-1
votes
1answer
25 views
Binary classification problem on imbalanced data
Literature mainly says that in general is a good idea to apply some technique to balance the two classes. For a Neural Net, what is most important here? The fact of having an imbalanced dataset (freq. ...
2
votes
1answer
38 views
When developing machine learning models, is the size of each class in the test set important?
I am thinking about the prospective application of a trained classifier in a real-world context. We know that when we do over/under-sampling to balance our dataset, we never touch the testing set as ...
1
vote
1answer
37 views
Ranking problem and imbalanced dataset
I know about the problems that imbalanced dataset will cause when we are working on classification problems. And I know the solution for that including undersampling and oversampling.
I have to work ...
0
votes
1answer
29 views
Which metric should I use for classifying an imbalace data with fewer labels for the negative class?
From reading, I understand that when we have fewer positive class labels, it is better to use precision or recall as the evaluation metric. Which metric should I use when we have fewer negative ...
1
vote
1answer
119 views
Macro and micro average for imbalanced binary classes
Micro and macro averaging are metrics for multi-class classification. However, for binary classification when data are imbalanced, it seems that micro and macro precision have different results. My ...
0
votes
0answers
20 views
how to interpret this 'lift chart'? prediction and true labels
i am trying to compare the prediction from my classifcation model and it's true label either 0 or 1.
...
5
votes
4answers
146 views
Is an $F_1$ score of 0.1 always bad?
I'm currently building a model to predict early mortgage delinquency (60+ days delinquent within 2 years of origination) for loans originating in 2018Q1. I will eventually train out-of-time (on loans ...
-1
votes
1answer
30 views
How Should I deal with my imbalanced binary target [closed]
I am trying to model my data with Python and i am having concerns about my binary target variable, because it has 90% cases falling in 0 and 10% of the cases falling in 1. I have tried upsampling my ...
0
votes
0answers
81 views
how to choose importance_type in lgbm.plot_importance
I have made a binary-classifier using lgbm.The classifier is made on unbalanced dataset.
I wanted to see the importance features of the model. There are two types of selecting importance_type -
...
-2
votes
1answer
25 views
How to achieve better accuracy of 90+ on a 3 class highly skewed dataset?
I have a 3 class dataset with very high imbalance classes:
class 1: 75000
class 2: 27000
class 3: 3000
With simple learning algorithms, accuracy is ...
0
votes
1answer
49 views
How to set a class_weight Dictionary for Random Forest?
I'm dealing with an unbalanced dataset, so I decided to use a weight dictionary for classification.
Documentation says that a weight dict must be defined as shown below:
https://imbalanced-learn.org/...
0
votes
1answer
28 views
Model accuracy: how to determine it?
I have some doubts regarding the approach to build a classifier such as Multinomial Naive Bayes or SVM. I will go through the steps to see if the approach is fine. I have not a lot of experience in ...
0
votes
2answers
59 views
What makes the validation set a good representative of the test set? [closed]
I am developing a classification model using an imbalanced dataset. I am trying to use different sampling techniques to improve the model performance.
For my baseline model, I defined an AdaBoost ...
1
vote
0answers
34 views
Image segmentation with large class imbalance leads to zero precision/recall
I have a binary semantic image classification problem where only very small parts of the images are positive, most of it is negative. In the training data I have a positive rate of around 0.023, which ...
5
votes
1answer
259 views
Why you shouldn't upsample before cross validation
I have an imbalanced dataset and I am trying different methods to address the data imbalance. I found this article that explains the correct way to cross validate when oversampling data using SMOTE ...
