Data Science

Questions tagged [class-imbalance]

Ask Question

The tag has no usage guidance.

133 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
3
votes
1answer
32 views

How to balance class weights correct for a CNN in Keras, given an unbalanced data set?

I want to use class weights for training a CNN with a imbalanced data set. The question arise if the sum of the weights of all examples have to stays the same? My previous plan was to use the ...
0
votes
1answer
26 views

Undersampling improvement F1-score

Im doing a 2-class classification project for an imbalanced data set. The imbalance is about 18%/82%. Im noticing a huge improvement in F1-score when I under-sample; from 16% without under-sampling to ...
1
vote
0answers
13 views

Equivalent procedure to Scikit-Learns class_weight=balanced in Keras?

I want to train a SVM and a CNN with the same unbalanced multiclass-dataset and want to compare the results. I use Scikit-Learn for the SVM and Keras for the CNN. My goal is that no class is ...
4
votes
3answers
45 views

What is the best metric to evaluate highly imbalanaced binary classifiction? (such as fraud detection in credit card)

What is the best metric to evaluate highly imbalanaced binary classifiction? (such as fraud detection in credit card? I have examining several metrics precision recall F1 lassification Report (macro ...
3
votes
2answers
34 views

macro average and weighted average meaning in classification_report

I use the "classification_report" from from sklearn.metrics import classification_report in order to evaluate the imbalanced binary classification ...
2
votes
1answer
55 views

Oversampling possible improvement

I am currently solving a classification problem for an imbalanced data set (approximately 17% of the minority class). I split the data using a stratified k-fold split from sklearn (Stratified shuffle ...
1
vote
2answers
59 views

Training model on a Balanced vs Imbalanced dataset?

Let's say that I have a 2-class classification problem where classes A & B have 10*N and ...
2
votes
1answer
38 views

How does one decide a dataset to be labelled 'Imbalanced'?

What, specifically, is the proportion that serves as the demarcation between balanced and imbalanced datasets in a binary classification problem as it is almost impossible to get a dataset with ...
0
votes
0answers
30 views

Why does a class weight fraction improve precision compared to under-sampling approach where precision drops?

I have an imbalanced data where the ratio between positive to negative samples is 1:3 (positive samples are 3 times higher than negative). For my case it is is important to have a higher precision (...
2
votes
1answer
46 views

What is the definition of imbalanced data set

I have thousands of data sources generating data from similar type of hardware. The different sources create different dynamics in the datasets though! Even though the features are the same the data ...
1
vote
1answer
44 views

Sequence to carry out data analysis?

I have a dataset with 4700 records and it's a classification problem. Proportion of classes is 33 and 67% few questions 1) does this proportion qualify dataset as imbalanced ? 2) should I do ...
1
vote
1answer
16 views

What are the standard parameters values for SMOTE technique?

I'm working on an imbalanced class data set (200 samples) with 2 classes, first class has 50 sample and second has 150 sample. My questions: When I use SMOTE technique on my data set my total ...
1
vote
0answers
25 views

How to deal with time-series imbalance classification data?

I want to predict the user to buy or not a product in next month in the e-comercial site. I mainly using the past 1-year data to predict it. But I found the training data is imbalanced, and the buy(1)...
0
votes
1answer
28 views

How much is the Class Imbalance Problem rates?

I'm working on a data set and wanted to know is there a standard rate about Class Imbalance problem or not? I have 47 samples in Class A and 150 Sample in class B , should I use Class Imbalance ...
3
votes
2answers
28 views

Does Sampling size matters in Multi classification Model

I am working on a multi class classification model where few of the class are with less data compare to other classes. I used random sampling technique to create a sample from the population keeping ...
1
vote
1answer
22 views

Adjust predicted probability after smote

i have an imbalance data set and I used smote to oversample the minority class and undersample the majority class. now, I want to check the test AUC using predict_proba of the model. I have two ...
2
votes
0answers
57 views

Data Augmentation techniques for classification of imbalanced time series datasets

Now I have a task to classify the imbalanced time series datasets using ML classifiers, such as Logistic Regression, Decision Tree, SVM, and KNN. I am not allowed to use the Deep Learning tools, such ...
0
votes
1answer
71 views

How do I run SMOTE on image data using the packages available?

I need to balance some image datasets, how do I use SMOTE variants or the imblearn SMOTE method with images? I can't figure out how to, since they seem to be working only with numerical datasets.
1
vote
1answer
14 views

Balancing data-sets for regression problems

Unbalanced data-sets are a well described problem for classification-problems. However, for regression similar problems can arise. An example is the data-set where target variable has a very ...
2
votes
2answers
32 views

model predicting probability close to 50 for positive classes in imbalanced training dataset

I have a binary classification model where I am predicting the positive class which is only 10% of whole training data set. The issue with this imbalanced data set is my model is predicting ...
0
votes
0answers
11 views

Class balancing in Weka

Hi I am using software fault prediction datasets, which has class values Yes (faulty) and No (non faulty). I have to do class balancing at ratio of 20-80% (faulty=20, non faulty=80). So for that,do I ...
0
votes
2answers
40 views

Imbalanced dataset - Undersampling & multiple classifiers [duplicate]

Let's suppose that my dataset in a classification problem looks like that: class A: 50000 observations class B: 2000 observations class C: 800 observations class D: 200 observations These are some ...
1
vote
2answers
93 views

Which scoring for GridSearchCV is best, when imbalanced multiclass dataset?

I have an unbalanced multiclass dataset (GTSRB) and want to optimize the hyperparameters of an SVM through GridSearchCV. I know that accuracy is not suitable for scoring in this case. Which evaluation ...
0
votes
0answers
18 views

Stratified sampling for large imbalanced data set

I have a very large data set (64 Million Rows) and I want to understand the best approach to sample the data for explorative data analysis before proceeding to perform any classification modeling. ...
0
votes
2answers
99 views

How to find whether a dataset is blanced or imbalanced?

I have few dataset to experiment classification(Multi-class). These datasets are about 400GB. I wanted to know whether the dataset is balanced or imbalanced. How to know that dataset is balance or ...
0
votes
3answers
261 views

How to Split And Resample Imbalanced Dataset Into Train, Validation and Test

I want to understand how to split the imbalanced data set with a binary target variable where 87% of the samples are negative and 13% of the samples are positive. Now, I know that you should always ...
1
vote
1answer
26 views

How to control the amount of positives in classification?

I have a basic, yet quite complex problem to solve right now. Let's say we have a training set of 20,000 samples in my training set, out of which 3 to 4% is flagged as "True", the rest is flagged as "...
2
votes
3answers
51 views

Highly Imbalanced dataset fro classes more than 200

I have a text dataset where I need to train a classifier to classify the titles into categories. The dataset shape is more than 575000. There are 256 target classes here. The problem is the dataset is ...
4
votes
3answers
132 views

Why did sampling boost the performance of my model?

I have an imbalanced dataset with 88 positive samples and 128575 negative samples. I was reluctant to over/undersample the data since it's a biological dataset and I didn't want to introduce synthetic ...
0
votes
0answers
18 views

Clustering in python when imbalanced data sets exist

I have a set of measurements with four features. Two features are continuous (time and distance) and two are discrete. We also know that the population consists of two groups. One is the minority ...
0
votes
1answer
210 views

SMOTE vs SMOTENC for binary classifier with categorical and numeric data

I have a problem that I am having trouble thoroughly understanding. I am using Xgboost for classification. My y is 0 or 1 (true or false). I have categorical and numeric features, so theoretically, I ...
0
votes
1answer
25 views

Why does downsampling leads classification to only predict one class?

I have a multi-class classification problem. It performs quite well but on the least represented classes it doesn't. Indeed, here is the distribution : And here are the classification results of my ...
1
vote
2answers
2k views

Weighted Binary Cross Entropy Loss — Keras Implementation

I have a binary segmentation problem with highly imbalanced data such that there are almost 60 class zero samples for every class one sample. To address this issue, I coded a simple weighted binary ...
0
votes
1answer
44 views

Evaluate imbalanced classification model on balanced testing sample

Why it would be too optimistic to compute presicion, recall and f1-score to evaluate a model trained for imbalanced classification on a balanced testing sample ?
3
votes
2answers
148 views

Is There a Way to Re-Calibrate Predicted Probabilities After Using Class Weights?

I have classification data with far more negative instances than positive instances. I have used class weights in my models and have achieved the discrimination I want but the predicted probabilities ...
0
votes
1answer
45 views

Combining 'class_weight' with SMOTE

This might sound a weird question, but I could not find enough details in sklearn documentation about 'class_weight'. Can we first oversample the dataset using SMOTE and then call the classifier with ...
0
votes
1answer
89 views

Choice of f1 score for highly imbalanced dataset?

I am confused whether to use f1 score with 'micro' average or 'macro' average for better evaluation. Given my dataset is highly imbalanced(600:100000)
2
votes
1answer
162 views

Difference between sklearn make_pipeline and imblearn make_pipeline

Can anybody please explain the difference between sklearn.pipeline.make_pipline and imblearn.pipeline.make_pipline.
1
vote
1answer
126 views

Oversampling only balances the training set, what about the testing set?

In a case of imbalanced data classification, I know that we only oversample the training set (to prevent data leakage from training to testing subsets), but what if there are no positive data points ...
2
votes
2answers
110 views

Resampling for imbalaced datasets: should testing set also be resampled?

Apologies for what is probably a basic question but I have not been able to find a definitive answer either in the literature or in the Internet. When dealing with an imbalanced dataset one possible ...
6
votes
2answers
74 views

Can we specify the number of data generated(minority class) using SMOTE?

I am trying to improve classification of imbalanced dataset creditcard fraud using SMOTE imbalanced_learn. But, in this it generates the data to 50%, can we give a specific number for the data to be ...
1
vote
1answer
170 views

ROC AUC score is better if test data is imbalanced

I have an imbalanced dataset and I'm using XGBoost to do binary classification. I used down sampling together with target and one hot encoding for train data. For ...
0
votes
0answers
29 views

GridSearch on imbalanced multi-class dataset

I have an imbalanced multi-class dataset (GTSRB) and would like to use GridSearch to determine the hyperparameters for an SVM. As metric for the evaluation I chose F1 with average macro. ...
1
vote
1answer
143 views

Choosing weights on random forest for imbalanced data with the aim to minimize false positives

I am currently dealing with a binary classification task on imbalanced data with the following distribution: ...
1
vote
1answer
197 views

Poor performance of regression model for imbalanced data

I am trying to train a neural network model to solve a regression problem. The specificity of my dataset is that it has something like an exponential distribution of target values (imbalanced). ...
2
votes
1answer
423 views

How does class_weight work in Decision Tree

The scikit-learn implementation of DecisionTreeClassifier has a parameter as class_weight. ...
3
votes
1answer
60 views

Machine Learning: Balanced training set but highly unbalanced prediction set? How to adjust?

I am trying to train a model to detect gender in a dataset of CEO speeches. Here are the datasets that I have: Final Dataset: 20K CEO voices analyzed (around 95% male) Testing dataset (?): 1K CEO ...
1
vote
0answers
79 views

Best strategy to build Classifier for Mixed Data with class imbalance

I have a dataset which contains : 94 numeric features + 56 categorical features I am trying to build a classifier to predict Target (disease/healthy). 2. Rows : 1812 3. Class imbalance ( Majority ...
0
votes
0answers
13 views

How to identify whether or not my dataset is sufficient enough to be learnt from any model at all?

I created my own bipartite dataset where 1 group consists of disease and another group consists of genes. The number of diseases is significantly lower that number of genes. Furthermore, one disease ...
0
votes
2answers
51 views

Low prediction/classification accuracy due to imbalance in data feeding

I am building the neural network for image analysis to do Chest Xray classification (Abnormal/Pass). The classification accuracy for abnormal Xray is low, I guess it is due to the lack of abnormal ...

1 2 3