Questions tagged [unbalanced-classes]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
2answers
39 views

Training model on a Balanced vs Imbalanced dataset?

Let's say that I have a 2-class classification problem where classes A & B have 10*N and ...
1
vote
1answer
39 views

Sequence to carry out data analysis?

I have a dataset with 4700 records and it's a classification problem. Proportion of classes is 33 and 67% few questions 1) does this proportion qualify dataset as imbalanced ? 2) should I do ...
0
votes
4answers
74 views

What do you call the ratio of positive to negative samples?

I am working with a binary classifier and I want to express the "balance" or "skewness" of the training data using a metric. I want to reflect this ratio in a report, like this: ...
1
vote
1answer
13 views

On assiging weights for unbalanced classes

Consider a dataset that will be split into train and test. The model will be learned using the train set and evaluated using the unseen test set. Now the dataset is unbalanced -- it contains more ...
0
votes
1answer
20 views

What are advantages of oversampling over changing threshold for unbalanced classes?

Let's say that I have unbalanced data set that has two classes, and I am using Random Forest to make my predictions. Random forest will be biased towards the majority class, which will cause low ...
0
votes
2answers
75 views

Class Imbalance and Cost-Sensitive Learning XGBoost

I'm fairly new to data science and machine learning and have been trying to read a bit more on methods like boosting for one of the projects I am working on. The investigator on this project is ...
1
vote
2answers
103 views

class_weight on sklearn's DecisionTreeClassifier

Can class_weight='balanced' on scikit-learn's DecisionTreeClassifier be interpreted as having identical duplicate data points for the minority classes? I know that doesn't work that way, class_weight ...
2
votes
2answers
25 views

Predicting positive/negative experience with very few labels and labels from only one class

I have video viewing data (length of session, nb of videos, etc), as well as if the user clicked on the like button. We can use the like button as a confirmation that the user had a positive viewing ...
0
votes
0answers
54 views

Ensure class balanced batches while hyperparameter tuning keras models with grid search

Ensuring class balanced batches while training keras models is possible using fit_generator method. I used imblearn.keras.BalancedBatchGenerator for that and it works fine! But i wanted to do that ...
3
votes
2answers
60 views

Which classifier performs better when using 'class_weight'?

I have used the 'class_weight' method to balance my multi-class classification problem, using Logistic Regression, Random Forest, and XGBoost classifiers. Among these three methods, logistic ...
2
votes
0answers
178 views

Target mean encoding worse than ordinal encoding with GBDT ( XGBoost, CatBoost )

I have a dataset of 23k rows of an unbalanced dataset 85/15 ratio, 10 variables ( 9 of which are categorical ) , i'm using CatBoost and XGBoost for a binary classification. I applied cv (5 iteration ...
1
vote
1answer
47 views

What is the purpose of 'oversampling' when the test set is still unbalanced?

I understand that both training and testing sets should have the same distribution and also understand that we should not touch the test set (in terms of oversampling). But we know that oversampling ...
5
votes
2answers
60 views

Can we specify the number of data generated(minority class) using SMOTE?

I am trying to improve classification of imbalanced dataset creditcard fraud using SMOTE imbalanced_learn. But, in this it generates the data to 50%, can we give a specific number for the data to be ...
0
votes
0answers
27 views

GridSearch on imbalanced multi-class dataset

I have an imbalanced multi-class dataset (GTSRB) and would like to use GridSearch to determine the hyperparameters for an SVM. As metric for the evaluation I chose F1 with average macro. ...
0
votes
1answer
217 views

SMOTE-NC does not help to oversample my mixed continuous/categorical dataset

When I use SMOTE-NC to oversample three classes of a 4-class classification problem, the Prec, Recall, and F1 metrics for minority classes are still VERY low (~3%). I have 32 categorical and 30 ...
0
votes
1answer
36 views

Random forest with zero precision for unbalanced test data

Apologies if this is a basic question. I have a very unbalanced dataset in which the records are labelled by one of two classes, class1 (negative class) and class2 (positive class): class 1: 1.5 ...
1
vote
1answer
39 views

Class Imbalence Problem even after Balancing Data

So I am training a neural network on a binary classification problem and my Case (1) and Controls (0) were imbalanced so I oversampled my cases so that that the training set was 0.5053 made up of ...
1
vote
1answer
35 views

Class balancing of the dataset

While performing the SMOTE for balancing the class data, what should be the proportion of both class? For instance, if we have 100 instances, what (%) should be the Yes class and what should be the No ...
1
vote
2answers
261 views

convert predict_proba results using class_weight in training

As my dataset is unbalanced(class 1: 5%, class 0: 95%) I have used class_weight="balanced" parameter to train a random forest classification model. In this way I penalize the misclassification of a ...
1
vote
0answers
52 views

SMOTE and oversampling with constraints

I'm trying to apply SMOTE to a dataset that has time-constraints. I have information about users visiting a website. For some features, there are time constraints, e.g having the first visit and the ...
1
vote
2answers
33 views

Determining threshold in an area with very few samples of positive label

I have a binary classification task where I want to either keep or discard samples. I have about a million samples, and about 1% should be kept. I want to discard as much as possible, but discarding ...
2
votes
0answers
600 views

Differences between class_weight and scale_pos weight in LightGBM

I have a very imbalanced dataset with the ratio of the positive samples to the negative samples being 1:496. The scoring metric is the f1 score and my desired model is LightGBM. I am using the sklearn ...
0
votes
0answers
28 views

Ignoring unlabeled data for a single class

I have a data set of transactions with a binary flag labeling each as fraud or not fraud. However, it can take up to 90 days for a transaction to reveal itself as fraudulent. Sometimes it happens in a ...
2
votes
5answers
286 views

Large no of categorical variables with large no of categories

I'm working on a binary classification problem where the dataset is slightly imbalanced (30% class 0 | 70% class 1). Most of my features are categorical with large number of categories. For example: ...
2
votes
2answers
125 views

When to consider a target Variable unbalanced? [duplicate]

i'm performing a binary classification task , and after cheking the target variable , i saw that i had 69% of 0's and 31% of 1's , so , my question is , do i have in this case a unbalanced target ...
0
votes
0answers
17 views

Practical interpretation of Precision-Recall AUC

I have a classifier with an AUC (PR) of 0.06 which I will use for a practical interpretation. My test set consists of three months of data with a total of 2,200,000 observations of which 0.03 are ...
0
votes
0answers
18 views

Unbalance label of the data on binary classification

I am training a model with an unbalanced dataset. I used downsampling to fix the imbalance. In the entire dataset, 1% of the data is 1 and the rest of it (99%) is 0. When I downsample, the data is 50% ...
1
vote
1answer
42 views

Trying to predict extreme values corresponding to rare events

I need some advice on methodology. I need to predict a numeric value (claim amount) being as good as possible on high values corresponding to rare events (corporal damage, technological disaster...). ...
6
votes
1answer
221 views

Imbalanced classes (balance of train, validation, and test)

1) I am currently trying to set up a feedforward neural network with highly imbalanced classes (binary classification) in which the number of observations of class 1 is very low (and the class of ...
0
votes
3answers
207 views

Unbalanced target variable in Orange, how do I balance it?

So I want to perform a predictive model to predict churn. I have 2 datasets, one with churn and the other without (so I can later perform predictions). The issue is that I think my Confusion matrix ...
0
votes
0answers
69 views

XGBoost multiclass class balancing using weight parameter [duplicate]

I have three classes in the target variable with representation ratios of, class A:0.5 class B:0.3 ...
2
votes
2answers
45 views

oversampling data with subclass

Oversampling of under-represented data is a way to combat class imbalance. For example, if we have a training data set with 100 data points of class A and 1000 data points of class B, we can over ...
1
vote
0answers
45 views

Kappa Goes up as Accuracy Goes Down

I have recently been trying to train a randomForest model on a binary outcome with a very uneven class split. 282 control ~82% 63 case ~18% There are a total of 147 predictors that I'm testing for ...
6
votes
1answer
175 views

How to compare two unsupervised anomaly detection algorithms on the same data-set?

I want to solve an anomaly detection problem on an unlabeled data-set. The only information about this problem is that the anomalies population is lower than 0.1%. It should be notice that the size of ...
1
vote
1answer
47 views

What are some possible reasons that your multiclass classifier is classifying alll the classes in a single class?

I have unbalanced classes. Group1 N = 140 Group2 N = 35 Group3 N = 30 I ran the code on this data and all the Groups got classified as Group1. I thought that since group1 is the majority group this ...
0
votes
0answers
119 views

Using logLoss as metric function for highly unbalanced dataset

ihave an highly unbalanced dataset and the caret pacjage only allows me to select accuracy or kappa as performance metric. Is it correct to use a mlogloss function to compute model performance? Do you ...
3
votes
1answer
219 views

Overfitting - how to detect it and reduce it?

I have a side project where I am doing credit scoring using R (sample size around 16k for train data and 4k for test data, and also another two 20k data batches for out-of-time validation) with ...
1
vote
0answers
135 views

Relation between using stratify and class weights for imbalanced classes

I'm working on a multi-class classification problem where the classes are imbalanced (70:25:5). Train-Test Split ...
2
votes
1answer
711 views

Why doesn't class weight resolve the imbalanced classification problem?

I know that in imbalanced classification, the classifier tends to predict all the test labels as larger class label, but if we use class weight in loss function, it would be reasonable to expect the ...
1
vote
1answer
257 views

Regression - Unbalanced Categorical Features

I have a data set that has some unbalanced categorical features. I would like to build a regression model to predict a label using machine learning (ML). How do I handle data imbalances in ...
1
vote
0answers
25 views

Unbalanced data set

Good evening everyone, I'd like to train a dataset for a multiclass classification. The problem is my dataset is very unbalanced. I mean, i have lots of elements from a class and few from another. I'...
1
vote
0answers
27 views

SMOTE caused my total nrows in train to fall to a very small proportion

I have a highly skewed dataset with minority class in target being just about 4%. I decided to apply SMOTE using library DMwR in R. Here is my target: ...
1
vote
1answer
44 views

Audio classification data balance

I'm trying to make a "car sound detector" I have the data from https://urbansounddataset.weebly.com/urbansound.html site. Which has 1000 labeled sounds for 10 different classes(Car sound, dog bark, ...
1
vote
2answers
144 views

Poor performance for unbalanced dataset

Consider a dataset A which has examples for training in a binary classification problem. I have used SVM and applied the weighted method (in MATLAB) since the ...
1
vote
2answers
131 views

Doubt to use accuracy or macro f1 measure in an unbalanced classification task

I have a multi-class classification task where the organizers said that the final results will be using the Accuracy measure. The provided data is unbalanced, and I don't have an idea about the test ...
0
votes
1answer
250 views

Predicting on real test set gives only very high probability for 1 for a very unbalanced data

Excuse me for this brief description of the problem, as I'm very bound on time, I'll try to sum up as much as I can. I have a multivariate time-series, that I trained using an RNN, there are periods ...
0
votes
1answer
107 views

Best way to deal with realistically imbalanced dataset for Regression problem

I have a dataset where each object has a label between 0-1. Objects with label = 1 are very common but those with label = 0 are very rare. I am interested in predicting the label in unseen data. NOTE:...
4
votes
3answers
2k views

using sklearn class weight to increase number of positive guesses in extremely unbalanced data set?

Hi I have a poorly correlated and unbalanced data set I have to work with. The set is 2 classes, 0 has 96,000 values and 1 has about 200. When I run random forest or other methods I get an output like:...
0
votes
1answer
57 views

Is there any way how to make samples balanced?

I have a dataset which consists of attributes on breakdown of machines.The target variable is machine status which are populated with ones and zeros. The distribution of ones and zeros are given below ...
1
vote
0answers
180 views

Increase Specificity of a model using SMOTE arguments from DMwR package in R when training data is unbalanced

I'm working on a binary classification problem and training data which I'm using is unbalanced. I used SMOTE function from DMwR ...