Questions tagged [class-imbalance]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
-2
votes
1answer
31 views

'numpy.ndarray' object has no attribute 'plot' [closed]

I am trying to balance my data set and using imblearn library for this but After performing fit operation when i try to see the data count in dependent variable its showing me below error ...
0
votes
0answers
8 views

Computing Rare values After SMOGN - Imbalanced Regression

I am dealing with a regression problem where I have the phenomenon "Imbalanced Regression". In my problem, the most relevant events are scarcely represented. In order for me to evaluate my models' ...
0
votes
1answer
19 views

Binary classification problem with imbalanced dataset, how to compare to random classifier

We have a very imbalanced dataset (2% of class 1). To the best of our knowledge, there is no baseline in the literature to the problem we want to solve - so we thought of comparing our performance to ...
2
votes
1answer
20 views

Kohen Kappa Coefficient of Naive Bayes with 62% overall accuracy is better than Logistic Regression with 98% accuracy?

I have been trying to evaluate my models used on fire systems dataset with a huge imbalance in the dataset. Most models failed to predict any true positives correctly however naive Bayes managed to do ...
1
vote
1answer
52 views

XGBoost multiclassification interpreting predicted probabilities

Let's consider an example. I have patient level data, their symptoms, reading from various medical tests. Based on that, I have built a binary classifier given patient data to classify if they are ...
1
vote
0answers
21 views

Generate a balanced batch with ImageDataGenerator() and flow_from_directory()

Hi I am new to python and deep learning. I am doing a multiclass classification. My 3-classes dataset is imbalanced, the classes take about 50%, 40%, and 20%. I am trying to generate mini batches with ...
2
votes
1answer
29 views

I have 3 graphs of a binary Logistic Regression that I want to understand better what is happening and learn of a strategy to make the model better

My problem is the following: I have a binary Logistic Regression model with a very imbalanced dataset that outputs the percentage of the prediction. As can be seen in the images, as the threshold is ...
4
votes
1answer
203 views

AUC ROC metric on a Kaggle competition

I am trying to learn data modeling by working on a dataset from Kaggle competition. As the competition was closed 2 years back, I am asking my question here. The competition uses AUC-ROC as the ...
0
votes
1answer
22 views

SVM SVC: Metric for parameter optimization on imbalanced data

I trained a multiclass SVC with RBF kernel on a down-sampled (and therefore balanced) dataset. Now I want to perform grid search to find best cost and gamma. What performance metric should I optimize ...
0
votes
1answer
38 views

Macro F1 result higher than accuracy for imbalanced dataset

In one of the research papers on fake news detection, the authors collected a fake news binary dataset (fake vs. real news) consists of 16,817 real articles and <...
1
vote
1answer
17 views

is there any rule to apply pca to the imbalance data? [closed]

Is there any rule to apply PCA to imbalanced data? (randomforest, xgboost) I used multiclass imbalance data to pca but the log-loss accuracy getting decrease any theoritical background of this?
0
votes
1answer
51 views

Imbalanced dataset - Positive majority class

My dataset consists of 150 patients where 50 are controls/healthy (negative) and 100 are sick (positive). If I want my model to have high sensitivity at hight specificity, in other words to have low ...
0
votes
1answer
24 views

imbalanced target dataset(multi class)

I have a multi-class prediction problem but the 300classes is imbalanced should I make it balance all 300 class will predict the better result? is there an easier method to do this job? if I'm using ...
0
votes
0answers
18 views

why we use “class_weight='balanced” as a classifier parameter?

Please I have imbalanced data and at first, I used SMOTE to oversample it, and it gives a good result, then I see some code used sometimes class_weight='balanced" inside of the classifier ( RL, DT, ...
2
votes
1answer
18 views

Can sampling like SMOTE/UP/DOWN applied on Validation set?

I am trying to predict classification problem. For that I have used Ranger, Xgboost and naive bayes. My Response class is imbalance . 92:8 ratio. My positive Response is only 8% of whole data. ...
3
votes
1answer
133 views

classification imbalance data - bias and class weight

This page shows a classification problem. They have used bias as well as bias along with class weights. What is difference between bias and weights? In some other techniques such as Random Forest we ...
1
vote
0answers
31 views

setting class weights for imbalanced dataset, how using EarlyStopping?

I want to train a CNN with Early Stopping (Keras). The data set is imbalanced, so I have set class_weights to 'balanced' like follows: ...
1
vote
0answers
29 views

How to estimate the accuracy on a large dataset?

Given that I have a deep learning model(handover from former colleague). For some reason, the train/dev set was missing. In my situation, I want to classify my dataset into 100 categories. The ...
4
votes
1answer
88 views

How to balance class weights correct for a CNN in Keras, given an unbalanced data set?

I want to use class weights for training a CNN with a imbalanced data set. The question arise if the sum of the weights of all examples have to stays the same? My previous plan was to use the ...
0
votes
1answer
38 views

Undersampling improvement F1-score

Im doing a 2-class classification project for an imbalanced data set. The imbalance is about 18%/82%. Im noticing a huge improvement in F1-score when I under-sample; from 16% without under-sampling to ...
1
vote
0answers
15 views

Equivalent procedure to Scikit-Learns class_weight=balanced in Keras?

I want to train a SVM and a CNN with the same unbalanced multiclass-dataset and want to compare the results. I use Scikit-Learn for the SVM and Keras for the CNN. My goal is that no class is ...
4
votes
3answers
58 views

What is the best metric to evaluate highly imbalanaced binary classifiction? (such as fraud detection in credit card)

What is the best metric to evaluate highly imbalanaced binary classifiction? (such as fraud detection in credit card? I have examining several metrics precision recall F1 lassification Report (macro ...
2
votes
2answers
259 views

macro average and weighted average meaning in classification_report

I use the "classification_report" from from sklearn.metrics import classification_report in order to evaluate the imbalanced binary classification ...
2
votes
1answer
58 views

Oversampling possible improvement

I am currently solving a classification problem for an imbalanced data set (approximately 17% of the minority class). I split the data using a stratified k-fold split from sklearn (Stratified shuffle ...
1
vote
2answers
83 views

Training model on a Balanced vs Imbalanced dataset?

Let's say that I have a 2-class classification problem where classes A & B have 10*N and ...
2
votes
1answer
46 views

How does one decide a dataset to be labelled 'Imbalanced'?

What, specifically, is the proportion that serves as the demarcation between balanced and imbalanced datasets in a binary classification problem as it is almost impossible to get a dataset with ...
0
votes
0answers
39 views

Why does a class weight fraction improve precision compared to under-sampling approach where precision drops?

I have an imbalanced data where the ratio between positive to negative samples is 1:3 (positive samples are 3 times higher than negative). For my case it is is important to have a higher precision (...
2
votes
1answer
47 views

What is the definition of imbalanced data set

I have thousands of data sources generating data from similar type of hardware. The different sources create different dynamics in the datasets though! Even though the features are the same the data ...
1
vote
1answer
44 views

Sequence to carry out data analysis?

I have a dataset with 4700 records and it's a classification problem. Proportion of classes is 33 and 67% few questions 1) does this proportion qualify dataset as imbalanced ? 2) should I do ...
1
vote
1answer
25 views

What are the standard parameters values for SMOTE technique?

I'm working on an imbalanced class data set (200 samples) with 2 classes, first class has 50 sample and second has 150 sample. My questions: When I use SMOTE technique on my data set my total ...
1
vote
0answers
37 views

How to deal with time-series imbalance classification data?

I want to predict the user to buy or not a product in next month in the e-comercial site. I mainly using the past 1-year data to predict it. But I found the training data is imbalanced, and the buy(1)...
0
votes
1answer
29 views

How much is the Class Imbalance Problem rates?

I'm working on a data set and wanted to know is there a standard rate about Class Imbalance problem or not? I have 47 samples in Class A and 150 Sample in class B , should I use Class Imbalance ...
3
votes
2answers
30 views

Does Sampling size matters in Multi classification Model

I am working on a multi class classification model where few of the class are with less data compare to other classes. I used random sampling technique to create a sample from the population keeping ...
1
vote
1answer
51 views

Adjust predicted probability after smote

i have an imbalance data set and I used smote to oversample the minority class and undersample the majority class. now, I want to check the test AUC using predict_proba of the model. I have two ...
2
votes
0answers
122 views

Data Augmentation techniques for classification of imbalanced time series datasets

Now I have a task to classify the imbalanced time series datasets using ML classifiers, such as Logistic Regression, Decision Tree, SVM, and KNN. I am not allowed to use the Deep Learning tools, such ...
0
votes
1answer
325 views

How do I run SMOTE on image data using the packages available?

I need to balance some image datasets, how do I use SMOTE variants or the imblearn SMOTE method with images? I can't figure out how to, since they seem to be working only with numerical datasets.
1
vote
1answer
15 views

Balancing data-sets for regression problems

Unbalanced data-sets are a well described problem for classification-problems. However, for regression similar problems can arise. An example is the data-set where target variable has a very ...
2
votes
2answers
33 views

model predicting probability close to 50 for positive classes in imbalanced training dataset

I have a binary classification model where I am predicting the positive class which is only 10% of whole training data set. The issue with this imbalanced data set is my model is predicting ...
0
votes
0answers
16 views

Class balancing in Weka

Hi I am using software fault prediction datasets, which has class values Yes (faulty) and No (non faulty). I have to do class balancing at ratio of 20-80% (faulty=20, non faulty=80). So for that,do I ...
0
votes
2answers
44 views

Imbalanced dataset - Undersampling & multiple classifiers [duplicate]

Let's suppose that my dataset in a classification problem looks like that: class A: 50000 observations class B: 2000 observations class C: 800 observations class D: 200 observations These are some ...
1
vote
2answers
181 views

Which scoring for GridSearchCV is best, when imbalanced multiclass dataset?

I have an unbalanced multiclass dataset (GTSRB) and want to optimize the hyperparameters of an SVM through GridSearchCV. I know that accuracy is not suitable for scoring in this case. Which evaluation ...
0
votes
0answers
28 views

Stratified sampling for large imbalanced data set

I have a very large data set (64 Million Rows) and I want to understand the best approach to sample the data for explorative data analysis before proceeding to perform any classification modeling. ...
0
votes
2answers
262 views

How to find whether a dataset is blanced or imbalanced?

I have few dataset to experiment classification(Multi-class). These datasets are about 400GB. I wanted to know whether the dataset is balanced or imbalanced. How to know that dataset is balance or ...
1
vote
3answers
777 views

How to Split And Resample Imbalanced Dataset Into Train, Validation and Test

I want to understand how to split the imbalanced data set with a binary target variable where 87% of the samples are negative and 13% of the samples are positive. Now, I know that you should always ...
1
vote
1answer
27 views

How to control the amount of positives in classification?

I have a basic, yet quite complex problem to solve right now. Let's say we have a training set of 20,000 samples in my training set, out of which 3 to 4% is flagged as "True", the rest is flagged as "...
2
votes
3answers
74 views

Highly Imbalanced dataset fro classes more than 200

I have a text dataset where I need to train a classifier to classify the titles into categories. The dataset shape is more than 575000. There are 256 target classes here. The problem is the dataset is ...
4
votes
3answers
209 views

Why did sampling boost the performance of my model?

I have an imbalanced dataset with 88 positive samples and 128575 negative samples. I was reluctant to over/undersample the data since it's a biological dataset and I didn't want to introduce synthetic ...
0
votes
0answers
21 views

Clustering in python when imbalanced data sets exist

I have a set of measurements with four features. Two features are continuous (time and distance) and two are discrete. We also know that the population consists of two groups. One is the minority ...
1
vote
1answer
477 views

SMOTE vs SMOTENC for binary classifier with categorical and numeric data

I have a problem that I am having trouble thoroughly understanding. I am using Xgboost for classification. My y is 0 or 1 (true or false). I have categorical and numeric features, so theoretically, I ...
0
votes
1answer
29 views

Why does downsampling leads classification to only predict one class?

I have a multi-class classification problem. It performs quite well but on the least represented classes it doesn't. Indeed, here is the distribution : And here are the classification results of my ...