Questions tagged [class-imbalance]

Questions referring to classifiers or classifying problems where some of the classes in the data are under-represented.

Filter by
Sorted by
Tagged with
0
votes
1answer
9 views

Hypertune xgboost to dealing with imbalanced dataset

My training data has extremely class imbalanced {0:872525,1:3335} with 100 features. I use xgboost to build classification model with bayessian optimisation to hypertune the model in range {learning ...
5
votes
3answers
143 views

Metric for label imbalance

I'm looking for a metric that can be used to quantify how imbalanced the labels are in a dataset. I'm not looking for a strategy to solve the imbalance problem, I just want to present how imbalanced ...
0
votes
1answer
33 views

Any pythonic way to auto determine imbalance class problem, specially in multiclass scenario?

A data is imbalanced if a target class proportions are unequal and typically, heavily biased. But, what is the exact measurement of this heavy bias? Before applying imbalance techniques (SMOTE, ADASYN,...
0
votes
1answer
34 views

Handling Imbalanced Datasets

I work in the medical domain, so class imbalance is the rule and not the exception. While I know Python has packages for class imbalance, I don't see an option in Orange for e.g. a SMOTE widget. I ...
0
votes
1answer
20 views

Class imbalance and statistics

I have a dataset with 5000 observations with class 0 and 300 observations with class 1. I would like to run some statistical analysis, for example on the average length of strings, the number of words,...
0
votes
0answers
16 views

Cost Function Binary Classification

I have imbalance dataset for binary classification problem. I want to create a custom cost function that takes into account not only the actual class and probability, but another variable "...
0
votes
0answers
26 views

Confusion regarding accuracy and individual class performance

Consider a three-class classification problem where avg_cm1 and avg_cm2 are two average confusion matrices across 3 folds from ...
2
votes
2answers
66 views

Undersampling for credit card fraud detection before or after Train/Test Split

I have a credit card dataset with 98% transactions are Non-Fraud and 2% are fraud. I have been trying to undersample the majotrity class before train and test split and get very good recall and ...
3
votes
1answer
171 views

Features selection in imbalanced dataset

I have some doubts regarding an analysis. I have a dataset with class imbalance. I am trying to investigate some information from that data, e.g., how many urls contain http or https protocols. My ...
1
vote
1answer
32 views

What does the oob decision function mean in random forest, how get class predictions from it, and calculating oob for unbalanced samples

I am interested in finding the OOB score for random forest using sklearn, when it is used for a binary classification task, and there are unbalanced samples. What does the oob decision function mean ...
1
vote
1answer
14 views

Which metric to use for evaluating a rating system

I have a system which gives a star rating of the quality of work(on scale of 1-5, 1 being extremely poor and 5 being exceptionally good). An expert labelled a test set with their ratings of quality of ...
4
votes
1answer
93 views

Why removing rows with NA values from the majority class improves model performance

I have an imbalanced dataset like so: df['y'].value_counts(normalize=True) * 100 ...
1
vote
1answer
25 views

Test set larger than train set [closed]

There is a two class dataset with 1121 values in total, having 230 from same class and 891 from the other class. The training set is choosen as 230+230=460 from both classes and the test set as the ...
2
votes
1answer
28 views

What does IBA mean in imblearn classification report?

imblearn is a python library for handling imbalanced data. A code for generating classification report is given below. ...
0
votes
0answers
9 views

Modelling probability of switching of label from 0 to 1 in next time period for multivariate time series data

I have a data set consisting of unbalanced panel data, i.e. longitudinal multivariate measurements on multiple individuals. I want to estimate a probability that the individual will become 1 in next ...
0
votes
1answer
40 views

In which situation should we consider a dataset as imbalanced?

I'm facing a problem about making a classification on a dataset. The target variable is binary (with 2 classes, 0 and 1). I have 8,161 samples in the training dataset. And for each class, I have: ...
1
vote
1answer
30 views

Strong overfitting accompanying strong class imbalance

I'm training an xgboost binary classification model. The data I have is around 600k and positive is only 0.1% of it. I tried to use all overfitting prevention techniques xgboost has to offer (tune eta,...
0
votes
0answers
20 views

How can I fix my classifier only predicting two classes, and do my metrics show that it is overfitting?

I have a relatively simple 16 feature neural network attempting to predict the outcome of a sports event as win, loss, draw, however regardless of the number of layers, or the number of nodes in said ...
1
vote
0answers
33 views

How do I handle class imbalance for text data when using pretrained models like BERT?

I have a skewed dataset consisting of samples of the form: Category 1 10000 Category 2 2000 Category 3 400 Category 4 300 Category 5 100 The dataset ...
1
vote
0answers
20 views

Follow up question regarding Upsampling for Imbalanced Data and the use of ADASYN instead of SMOTE

I have a follow-up question regarding this topic. I have been working on a project predicting success(1) or failure(0) for organizations by using the Decision Tree and Random Forest algorithms. My ...
1
vote
1answer
27 views

Is it advisable to merge similar datasets to improve model accuracy?

I'm trying to build a classifier that would help me classify whether a statement collected from Reddit is bullish, bearish or neutral. To this end, I have hand-labelled a fairly small dataset of 2500 ...
2
votes
1answer
52 views

How to split up my dataset in a train and testset, in order to prevent data leakage?

I realize that this could be considered a duplicate of this question, Is using samples from the same person in both trainset and testset considers being a data leakage?, where it is stated that "...
2
votes
1answer
43 views

Calibrating probability thresholds for multiclass classification

I have built a network for the classification of three classes. The network consists of a CNN followed by two fully-connected layers. The CNN consists of convolutional layers, followed by batch ...
0
votes
1answer
17 views

Many questions training unbalanced and duplicated data

I'm a DS student. I have like 30.000 of bank statements, all labeled with a specific category(cat1, cat2, ...). With that data I'm trying to train a classification model but I found several problems: ...
2
votes
2answers
54 views

Is the PR AUC invariant under label flip?

The ROC-AUC curve is invariant under a flip of the labels. I don't know if its a famous result so I will give the proof below. My question is if the PR-AUC curve also has this property. I have not ...
-1
votes
1answer
22 views

using average precision as metric for imbalanced problem (learning curve example) [closed]

I have an imbalanced problem (2% target class) and therefore need an appropriate metric - so I chose average_precision. My code: ...
0
votes
0answers
29 views

Overfitting in imbalanced dataset

I am working on a dataset related to an insurance company and the objective is to predict if the insurance buyer will claim their travel insurance or not. Training data: https://raw.githubusercontent....
2
votes
1answer
58 views

Cross validation for unbalanced dataset using Orange data mining tool

I am using the Orange data mining tool to build and analyze models (decision tree, ANN, ...) predicting customer churn. As this is an imbalanced class problem (10% churn, 90% not churn), I need to ...
0
votes
0answers
10 views

Classification model performance - metrics for getting number in each class correct?

I'm fairly new to predictive modelling, so apologies if this is a stupid question. I am working on a classification problem (predicting if customers commit fraud or not), and have been comparing a few ...
0
votes
2answers
34 views

Training binary classifier on only one data point ( Theoritical question)

Say, I'm training a binary classifier to classify Dog vs Cat. Now, say I train my model only on one imagee ( cat). Now I mirror this cat image that I used to train my model. Now on the mirror image I ...
0
votes
0answers
15 views

Data leakage when setting class_weight to tackle imbalanced time series data?

I'm using a random forest classifier from sklearn to predict whether a stock's return for the next period is greater than a certain threshold (say -2%), so negative is 0 and positive is 1, a binary ...
1
vote
1answer
33 views

Labels as features in anomaly detection

I have a dataset born to solve a classification problem. Due to the imbalances of the Y, i choose to move to an anomaly detection task. Should I use the Y i have inside the anomaly detection model as ...
0
votes
0answers
16 views

How many instances should be synthesized for each class when using over-sampling techniques?

As for an imbalanced multi-class dataset, how many instances should be synthesized for each class if we use over-sampling techniques such as SMOTE? For example, there is 4 class including 'A', 'B', 'C'...
2
votes
1answer
42 views

How to tune an weighted voting ensemble method?

I am working on kidney cancer patients' data with 5 unbalanced labels. These codes are contained of Normalization, Oversampling on Feature Engineering part. A list of 9 ordinary Machine Learning ...
0
votes
0answers
17 views

Using accuracy metric during training for unbalanced multiclass classification

I am training a convolutional neural network and the sensitivity and precision of the minority class is what is most important to me. I am using 10-Fold cross validation, and the test fold is ...
-1
votes
1answer
25 views

Binary classification problem on imbalanced data

Literature mainly says that in general is a good idea to apply some technique to balance the two classes. For a Neural Net, what is most important here? The fact of having an imbalanced dataset (freq. ...
2
votes
1answer
38 views

When developing machine learning models, is the size of each class in the test set important?

I am thinking about the prospective application of a trained classifier in a real-world context. We know that when we do over/under-sampling to balance our dataset, we never touch the testing set as ...
1
vote
1answer
37 views

Ranking problem and imbalanced dataset

I know about the problems that imbalanced dataset will cause when we are working on classification problems. And I know the solution for that including undersampling and oversampling. I have to work ...
0
votes
1answer
29 views

Which metric should I use for classifying an imbalace data with fewer labels for the negative class?

From reading, I understand that when we have fewer positive class labels, it is better to use precision or recall as the evaluation metric. Which metric should I use when we have fewer negative ...
1
vote
1answer
119 views

Macro and micro average for imbalanced binary classes

Micro and macro averaging are metrics for multi-class classification. However, for binary classification when data are imbalanced, it seems that micro and macro precision have different results. My ...
0
votes
0answers
20 views

how to interpret this 'lift chart'? prediction and true labels

i am trying to compare the prediction from my classifcation model and it's true label either 0 or 1. ...
5
votes
4answers
146 views

Is an $F_1$ score of 0.1 always bad?

I'm currently building a model to predict early mortgage delinquency (60+ days delinquent within 2 years of origination) for loans originating in 2018Q1. I will eventually train out-of-time (on loans ...
-1
votes
1answer
30 views

How Should I deal with my imbalanced binary target [closed]

I am trying to model my data with Python and i am having concerns about my binary target variable, because it has 90% cases falling in 0 and 10% of the cases falling in 1. I have tried upsampling my ...
0
votes
0answers
81 views

how to choose importance_type in lgbm.plot_importance

I have made a binary-classifier using lgbm.The classifier is made on unbalanced dataset. I wanted to see the importance features of the model. There are two types of selecting importance_type - ...
-2
votes
1answer
25 views

How to achieve better accuracy of 90+ on a 3 class highly skewed dataset?

I have a 3 class dataset with very high imbalance classes: class 1: 75000 class 2: 27000 class 3: 3000 With simple learning algorithms, accuracy is ...
0
votes
1answer
49 views

How to set a class_weight Dictionary for Random Forest?

I'm dealing with an unbalanced dataset, so I decided to use a weight dictionary for classification. Documentation says that a weight dict must be defined as shown below: https://imbalanced-learn.org/...
0
votes
1answer
28 views

Model accuracy: how to determine it?

I have some doubts regarding the approach to build a classifier such as Multinomial Naive Bayes or SVM. I will go through the steps to see if the approach is fine. I have not a lot of experience in ...
0
votes
2answers
59 views

What makes the validation set a good representative of the test set? [closed]

I am developing a classification model using an imbalanced dataset. I am trying to use different sampling techniques to improve the model performance. For my baseline model, I defined an AdaBoost ...
1
vote
0answers
34 views

Image segmentation with large class imbalance leads to zero precision/recall

I have a binary semantic image classification problem where only very small parts of the images are positive, most of it is negative. In the training data I have a positive rate of around 0.023, which ...
5
votes
1answer
259 views

Why you shouldn't upsample before cross validation

I have an imbalanced dataset and I am trying different methods to address the data imbalance. I found this article that explains the correct way to cross validate when oversampling data using SMOTE ...

1
2 3 4 5
8