Questions tagged [class-imbalance]
The class-imbalance tag has no usage guidance.
127
questions
1
vote
2answers
39 views
Training model on a Balanced vs Imbalanced dataset?
Let's say that I have a 2-class classification problem where classes A & B have 10*N and ...
2
votes
1answer
33 views
How does one decide a dataset to be labelled 'Imbalanced'?
What, specifically, is the proportion that serves as the demarcation between balanced and imbalanced datasets in a binary classification problem as it is almost impossible to get a dataset with ...
0
votes
0answers
18 views
Why does a class weight fraction improve precision compared to under-sampling approach where precision drops?
I have an imbalanced data where the ratio between positive to negative samples is 1:3 (positive samples are 3 times higher than negative). For my case it is is important to have a higher precision (...
2
votes
1answer
40 views
What is the definition of imbalanced data set
I have thousands of data sources generating data from similar type of hardware. The different sources create different dynamics in the datasets though!
Even though the features are the same the data ...
1
vote
1answer
39 views
Sequence to carry out data analysis?
I have a dataset with 4700 records and it's a classification problem. Proportion of classes is 33 and 67%
few questions
1) does this proportion qualify dataset as imbalanced ?
2) should I do ...
1
vote
1answer
12 views
What are the standard parameters values for SMOTE technique?
I'm working on an imbalanced class data set (200 samples) with 2 classes, first class has 50 sample and second has 150 sample.
My questions:
When I use SMOTE technique on my data set my total ...
1
vote
0answers
19 views
How to deal with time-series imbalance classification data?
I want to predict the user to buy or not a product in next month in the e-comercial site. I mainly using the past 1-year data to predict it. But I found the training data is imbalanced, and the buy(1)...
0
votes
1answer
26 views
How much is the Class Imbalance Problem rates?
I'm working on a data set and wanted to know is there a standard rate about Class Imbalance problem or not?
I have 47 samples in Class A and 150 Sample in class B , should I use Class Imbalance ...
3
votes
2answers
27 views
Does Sampling size matters in Multi classification Model
I am working on a multi class classification model where few of the class are with less data compare to other classes.
I used random sampling technique to create a sample from the population keeping ...
1
vote
1answer
17 views
Adjust predicted probability after smote
i have an imbalance data set and I used smote to oversample the minority class and undersample the majority class. now, I want to check the test AUC using predict_proba of the model.
I have two ...
2
votes
0answers
31 views
Data Augmentation techniques for classification of imbalanced time series datasets
Now I have a task to classify the imbalanced time series datasets using ML classifiers, such as Logistic Regression, Decision Tree, SVM, and KNN. I am not allowed to use the Deep Learning tools, such ...
0
votes
1answer
28 views
How do I run SMOTE on image data using the packages available?
I need to balance some image datasets, how do I use SMOTE variants or the imblearn SMOTE method with images? I can't figure out how to, since they seem to be working only with numerical datasets.
1
vote
1answer
14 views
Balancing data-sets for regression problems
Unbalanced data-sets are a well described problem for classification-problems.
However, for regression similar problems can arise. An example is the data-set where target variable has a very ...
2
votes
2answers
30 views
model predicting probability close to 50 for positive classes in imbalanced training dataset
I have a binary classification model where I am predicting the positive class which is only 10% of whole training data set. The issue with this imbalanced data set is my model is predicting ...
0
votes
0answers
10 views
Class balancing in Weka
Hi I am using software fault prediction datasets, which has class values Yes (faulty) and No (non faulty). I have to do class balancing at ratio of 20-80% (faulty=20, non faulty=80).
So for that,do I ...
0
votes
2answers
39 views
Imbalanced dataset - Undersampling & multiple classifiers [duplicate]
Let's suppose that my dataset in a classification problem looks like that:
class A: 50000 observations
class B: 2000 observations
class C: 800 observations
class D: 200 observations
These are some ...
1
vote
2answers
72 views
Which scoring for GridSearchCV is best, when imbalanced multiclass dataset?
I have an unbalanced multiclass dataset (GTSRB) and want to optimize the hyperparameters of an SVM through GridSearchCV.
I know that accuracy is not suitable for scoring in this case.
Which evaluation ...
0
votes
0answers
16 views
Stratified sampling for large imbalanced data set
I have a very large data set (64 Million Rows) and I want to understand the best approach to sample the data for explorative data analysis before proceeding to perform any classification modeling.
...
0
votes
2answers
67 views
How to find whether a dataset is blanced or imbalanced?
I have few dataset to experiment classification(Multi-class). These datasets are about 400GB. I wanted to know whether the dataset is balanced or imbalanced. How to know that dataset is balance or ...
0
votes
3answers
144 views
How to Split And Resample Imbalanced Dataset Into Train, Validation and Test
I want to understand how to split the imbalanced data set with a binary target variable where 87% of the samples are negative and 13% of the samples are positive. Now, I know that you should always ...
1
vote
1answer
26 views
How to control the amount of positives in classification?
I have a basic, yet quite complex problem to solve right now. Let's say we have a training set of 20,000 samples in my training set, out of which 3 to 4% is flagged as "True", the rest is flagged as "...
2
votes
3answers
46 views
Highly Imbalanced dataset fro classes more than 200
I have a text dataset where I need to train a classifier to classify the titles into categories. The dataset shape is more than 575000. There are 256 target classes here. The problem is the dataset is ...
3
votes
3answers
105 views
Why did sampling boost the performance of my model?
I have an imbalanced dataset with 88 positive samples and 128575 negative samples. I was reluctant to over/undersample the data since it's a biological dataset and I didn't want to introduce synthetic ...
0
votes
0answers
16 views
Clustering in python when imbalanced data sets exist
I have a set of measurements with four features.
Two features are continuous (time and distance) and two are discrete.
We also know that the population consists of two groups. One is the minority ...
0
votes
1answer
115 views
SMOTE vs SMOTENC for binary classifier with categorical and numeric data
I have a problem that I am having trouble thoroughly understanding.
I am using Xgboost for classification. My y is 0 or 1 (true or false). I have categorical and numeric features, so theoretically, I ...
0
votes
1answer
22 views
Why does downsampling leads classification to only predict one class?
I have a multi-class classification problem. It performs quite well but on the least represented classes it doesn't. Indeed, here is the distribution :
And here are the classification results of my ...
1
vote
2answers
1k views
Weighted Binary Cross Entropy Loss — Keras Implementation
I have a binary segmentation problem with highly imbalanced data such that there are almost 60 class zero samples for every class one sample. To address this issue, I coded a simple weighted binary ...
0
votes
1answer
43 views
Evaluate imbalanced classification model on balanced testing sample
Why it would be too optimistic to compute presicion, recall and f1-score to evaluate a model trained for imbalanced classification on a balanced testing sample ?
3
votes
2answers
139 views
Is There a Way to Re-Calibrate Predicted Probabilities After Using Class Weights?
I have classification data with far more negative instances than positive instances. I have used class weights in my models and have achieved the discrimination I want but the predicted probabilities ...
0
votes
1answer
38 views
Combining 'class_weight' with SMOTE
This might sound a weird question, but I could not find enough details in sklearn documentation about 'class_weight'. Can we first oversample the dataset using SMOTE and then call the classifier with ...
0
votes
1answer
70 views
Choice of f1 score for highly imbalanced dataset?
I am confused whether to use f1 score with 'micro' average or 'macro' average for better evaluation.
Given my dataset is highly imbalanced(600:100000)
2
votes
1answer
114 views
Difference between sklearn make_pipeline and imblearn make_pipeline
Can anybody please explain the difference between sklearn.pipeline.make_pipline and imblearn.pipeline.make_pipline.
1
vote
1answer
84 views
Oversampling only balances the training set, what about the testing set?
In a case of imbalanced data classification, I know that we only oversample the training set (to prevent data leakage from training to testing subsets), but what if there are no positive data points ...
2
votes
2answers
76 views
Resampling for imbalaced datasets: should testing set also be resampled?
Apologies for what is probably a basic question but I have not been able to find a definitive answer either in the literature or in the Internet.
When dealing with an imbalanced dataset one possible ...
5
votes
2answers
60 views
Can we specify the number of data generated(minority class) using SMOTE?
I am trying to improve classification of imbalanced dataset creditcard fraud using SMOTE imbalanced_learn. But, in this it generates the data to 50%, can we give a specific number for the data to be ...
1
vote
1answer
142 views
ROC AUC score is better if test data is imbalanced
I have an imbalanced dataset and I'm using XGBoost to do binary classification. I used down sampling together with target and one hot encoding for train data. For ...
0
votes
0answers
27 views
GridSearch on imbalanced multi-class dataset
I have an imbalanced multi-class dataset (GTSRB) and would like to use GridSearch to determine the hyperparameters for an SVM. As metric for the evaluation I chose F1 with average macro.
...
1
vote
1answer
119 views
Choosing weights on random forest for imbalanced data with the aim to minimize false positives
I am currently dealing with a binary classification task on imbalanced data with the following distribution:
...
1
vote
1answer
152 views
Poor performance of regression model for imbalanced data
I am trying to train a neural network model to solve a regression problem. The specificity of my dataset is that it has something like an exponential distribution of target values (imbalanced). ...
2
votes
1answer
293 views
How does class_weight work in Decision Tree
The scikit-learn implementation of DecisionTreeClassifier has a parameter as class_weight.
...
3
votes
1answer
59 views
Machine Learning: Balanced training set but highly unbalanced prediction set? How to adjust?
I am trying to train a model to detect gender in a dataset of CEO speeches. Here are the datasets that I have:
Final Dataset: 20K CEO voices analyzed (around 95% male)
Testing dataset (?): 1K CEO ...
1
vote
0answers
71 views
Best strategy to build Classifier for Mixed Data with class imbalance
I have a dataset which contains : 94 numeric features + 56 categorical features
I am trying to build a classifier to predict Target (disease/healthy).
2. Rows : 1812
3. Class imbalance ( Majority ...
0
votes
0answers
13 views
How to identify whether or not my dataset is sufficient enough to be learnt from any model at all?
I created my own bipartite dataset where 1 group consists of disease and another group consists of genes.
The number of diseases is significantly lower that number of genes. Furthermore, one disease ...
0
votes
2answers
41 views
Low prediction/classification accuracy due to imbalance in data feeding
I am building the neural network for image analysis to do Chest Xray classification (Abnormal/Pass).
The classification accuracy for abnormal Xray is low, I guess it is due to the lack of abnormal ...
1
vote
0answers
29 views
Problem On Class Imbalanced Data
I am getting an F-Score of 0.99 on the train_test_split data, but only getting 0.40 for a competition's test data. I am oversampling with random forest (after ...
3
votes
1answer
44 views
Which classification algorithms are negatively affected by class imbalances?
I've seen a few posts and papers floating around the web (mostly those related to over/undersampling, SMOTE, and cost-sensitive training) that, when discussing class imbalance, specify that certain ...
1
vote
2answers
88 views
Clustering on imbalanced data that has high correlation
I am clustering images of two categories, but for the purposes of the experiment, I do not know the labels i.e. this is an unsupervised problem. Via ...
0
votes
0answers
63 views
Metrics parameter in model.compile function in Keras for imbalanced data set
Accuracy for an imbalanced data set is not relevant and therefore I use precision and recall to evaluate my model. However, whenever I train a model in Keras a metrics parameter must be specified in ...
1
vote
0answers
28 views
Different result between Rapidminer and Python imblearn
I'm currently working on imbalanced classification problem. However i found different result between SMOTE in rapidminer and SMOTE in imblearn (python). rapidminer SMOTE give 15-20% improvement on ...
0
votes
2answers
58 views
A robust metric in the presence of class imbalance
When evaluating the performance of a multiclass classification problem, on a highly imbalanced dataset, what is the most robust metric for this purpose?
I read a paper that states:
"Average ...
