Questions tagged [unbalanced-classes]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
14 views

How to impute missing values for hour of day?

I have a dataset containing a certain amount of bookings, no shows and cancellations. We assume that a cancellation less than 2 days is the same as a no show (since you did not have the time to ...
1
vote
0answers
21 views

Generate a balanced batch with ImageDataGenerator() and flow_from_directory()

Hi I am new to python and deep learning. I am doing a multiclass classification. My 3-classes dataset is imbalanced, the classes take about 50%, 40%, and 20%. I am trying to generate mini batches with ...
1
vote
2answers
39 views

Weights for unbalanced classification

I'm working with an unbalanced classification problem, in which the target variable contains: np.bincount(y_train) array([151953, 13273]) i.e. ...
2
votes
0answers
19 views

Balancing the dataset using imblearn undersampling, oversampling and combine?

I have the imbalanced dataset data['Class'].value_counts() Out[22]: 0 137757 1 4905 Name: Class, dtype: int64 ...
0
votes
1answer
17 views

Over/Under Sampling for Multi-classification

I'm trying to apply xgboost and random forest for over and under sampling For imbalanced data: train shape -> (199991, 23) However, reverse my expectation. ...
1
vote
2answers
35 views

Different performance for splitting into test/train data vs. using cross-validation

I am training a linear model using the following scikit-learn setup: ...
1
vote
0answers
13 views

Multiclass Classification Task - Performance on Each Class Compared to Chance?

As a part of a classification task, a classifier has decided whether different books belong to class A, B or C (which are imbalanced) by looking at certain feature of the book. I have calculated ...
0
votes
0answers
18 views

why we use “class_weight='balanced” as a classifier parameter?

Please I have imbalanced data and at first, I used SMOTE to oversample it, and it gives a good result, then I see some code used sometimes class_weight='balanced" inside of the classifier ( RL, DT, ...
2
votes
0answers
20 views

Unbalanced data set - how to optimize hyperparams via grid search?

I would like to optimize the hyperparameters C and Gamma of an SVC by using grid search for an unbalanced data set. So far I have used class_weights='balanced' and selected the best hyperparameters ...
4
votes
1answer
88 views

How to balance class weights correct for a CNN in Keras, given an unbalanced data set?

I want to use class weights for training a CNN with a imbalanced data set. The question arise if the sum of the weights of all examples have to stays the same? My previous plan was to use the ...
2
votes
1answer
32 views

Using Majority Class to Predict Minority Class

Suppose I want to train a binary model in order to predict the probability of who will buy a personal loan and in the dataset only 5 percent of the examples are people who marked as bought a personal ...
0
votes
1answer
68 views

splitting into train test by train_test_split of float values?

How to split into train test by train_test_split of float values ? I used LabelEncoder but I have about 300K lines and when I used the cross_val I saw ...
1
vote
0answers
15 views

Equivalent procedure to Scikit-Learns class_weight=balanced in Keras?

I want to train a SVM and a CNN with the same unbalanced multiclass-dataset and want to compare the results. I use Scikit-Learn for the SVM and Keras for the CNN. My goal is that no class is ...
0
votes
0answers
44 views

undersampling problem in imblanced dataset ValueError: Unknown label type: 'continuous'

I would like to undersampling the data but I encounter the following error? ...
0
votes
0answers
13 views

Interaction with unseen data (Generalization and evaluation the performance on unseen data in supervised and unsupevised learning methods)

How to generalizes model and performs on unseen data for a highly imbalanced binary classification problem (99.827%,0.173%)? 1-When using supervised learning methods such as logistic reg, RFs, ...
1
vote
2answers
83 views

Training model on a Balanced vs Imbalanced dataset?

Let's say that I have a 2-class classification problem where classes A & B have 10*N and ...
1
vote
1answer
44 views

Sequence to carry out data analysis?

I have a dataset with 4700 records and it's a classification problem. Proportion of classes is 33 and 67% few questions 1) does this proportion qualify dataset as imbalanced ? 2) should I do ...
0
votes
4answers
99 views

What do you call the ratio of positive to negative samples?

I am working with a binary classifier and I want to express the "balance" or "skewness" of the training data using a metric. I want to reflect this ratio in a report, like this: ...
1
vote
1answer
17 views

On assiging weights for unbalanced classes

Consider a dataset that will be split into train and test. The model will be learned using the train set and evaluated using the unseen test set. Now the dataset is unbalanced -- it contains more ...
1
vote
1answer
32 views

What are advantages of oversampling over changing threshold for unbalanced classes?

Let's say that I have unbalanced data set that has two classes, and I am using Random Forest to make my predictions. Random forest will be biased towards the majority class, which will cause low ...
0
votes
2answers
310 views

Class Imbalance and Cost-Sensitive Learning XGBoost

I'm fairly new to data science and machine learning and have been trying to read a bit more on methods like boosting for one of the projects I am working on. The investigator on this project is ...
2
votes
2answers
320 views

class_weight on sklearn's DecisionTreeClassifier

Can class_weight='balanced' on scikit-learn's DecisionTreeClassifier be interpreted as having identical duplicate data points for the minority classes? I know that doesn't work that way, class_weight ...
2
votes
2answers
30 views

Predicting positive/negative experience with very few labels and labels from only one class

I have video viewing data (length of session, nb of videos, etc), as well as if the user clicked on the like button. We can use the like button as a confirmation that the user had a positive viewing ...
1
vote
0answers
153 views

Ensure class balanced batches while hyperparameter tuning keras models with grid search

Ensuring class balanced batches while training keras models is possible using fit_generator method. I used imblearn.keras.BalancedBatchGenerator for that and it works fine! But i wanted to do that ...
3
votes
2answers
70 views

Which classifier performs better when using 'class_weight'?

I have used the 'class_weight' method to balance my multi-class classification problem, using Logistic Regression, Random Forest, and XGBoost classifiers. Among these three methods, logistic ...
3
votes
0answers
280 views

Target mean encoding worse than ordinal encoding with GBDT ( XGBoost, CatBoost )

I have a dataset of 23k rows of an unbalanced dataset 85/15 ratio, 10 variables ( 9 of which are categorical ) , i'm using CatBoost and XGBoost for a binary classification. I applied cv (5 iteration ...
1
vote
1answer
51 views

What is the purpose of 'oversampling' when the test set is still unbalanced?

I understand that both training and testing sets should have the same distribution and also understand that we should not touch the test set (in terms of oversampling). But we know that oversampling ...
6
votes
2answers
85 views

Can we specify the number of data generated(minority class) using SMOTE?

I am trying to improve classification of imbalanced dataset creditcard fraud using SMOTE imbalanced_learn. But, in this it generates the data to 50%, can we give a specific number for the data to be ...
0
votes
0answers
34 views

GridSearch on imbalanced multi-class dataset

I have an imbalanced multi-class dataset (GTSRB) and would like to use GridSearch to determine the hyperparameters for an SVM. As metric for the evaluation I chose F1 with average macro. ...
0
votes
1answer
424 views

SMOTE-NC does not help to oversample my mixed continuous/categorical dataset

When I use SMOTE-NC to oversample three classes of a 4-class classification problem, the Prec, Recall, and F1 metrics for minority classes are still VERY low (~3%). I have 32 categorical and 30 ...
0
votes
1answer
67 views

Random forest with zero precision for unbalanced test data

Apologies if this is a basic question. I have a very unbalanced dataset in which the records are labelled by one of two classes, class1 (negative class) and class2 (positive class): class 1: 1.5 ...
1
vote
1answer
51 views

Class Imbalence Problem even after Balancing Data

So I am training a neural network on a binary classification problem and my Case (1) and Controls (0) were imbalanced so I oversampled my cases so that that the training set was 0.5053 made up of ...
1
vote
1answer
38 views

Class balancing of the dataset

While performing the SMOTE for balancing the class data, what should be the proportion of both class? For instance, if we have 100 instances, what (%) should be the Yes class and what should be the No ...
4
votes
2answers
616 views

convert predict_proba results using class_weight in training

As my dataset is unbalanced(class 1: 5%, class 0: 95%) I have used class_weight="balanced" parameter to train a random forest classification model. In this way I penalize the misclassification of a ...
1
vote
0answers
68 views

SMOTE and oversampling with constraints

I'm trying to apply SMOTE to a dataset that has time-constraints. I have information about users visiting a website. For some features, there are time constraints, e.g having the first visit and the ...
1
vote
2answers
36 views

Determining threshold in an area with very few samples of positive label

I have a binary classification task where I want to either keep or discard samples. I have about a million samples, and about 1% should be kept. I want to discard as much as possible, but discarding ...
3
votes
1answer
1k views

Differences between class_weight and scale_pos weight in LightGBM

I have a very imbalanced dataset with the ratio of the positive samples to the negative samples being 1:496. The scoring metric is the f1 score and my desired model is LightGBM. I am using the sklearn ...
0
votes
0answers
28 views

Ignoring unlabeled data for a single class

I have a data set of transactions with a binary flag labeling each as fraud or not fraud. However, it can take up to 90 days for a transaction to reveal itself as fraudulent. Sometimes it happens in a ...
2
votes
5answers
766 views

Large no of categorical variables with large no of categories

I'm working on a binary classification problem where the dataset is slightly imbalanced (30% class 0 | 70% class 1). Most of my features are categorical with large number of categories. For example: ...
2
votes
2answers
211 views

When to consider a target Variable unbalanced? [duplicate]

i'm performing a binary classification task , and after cheking the target variable , i saw that i had 69% of 0's and 31% of 1's , so , my question is , do i have in this case a unbalanced target ...
0
votes
0answers
18 views

Practical interpretation of Precision-Recall AUC

I have a classifier with an AUC (PR) of 0.06 which I will use for a practical interpretation. My test set consists of three months of data with a total of 2,200,000 observations of which 0.03 are ...
1
vote
1answer
44 views

Trying to predict extreme values corresponding to rare events

I need some advice on methodology. I need to predict a numeric value (claim amount) being as good as possible on high values corresponding to rare events (corporal damage, technological disaster...). ...
6
votes
1answer
316 views

Imbalanced classes (balance of train, validation, and test)

1) I am currently trying to set up a feedforward neural network with highly imbalanced classes (binary classification) in which the number of observations of class 1 is very low (and the class of ...
0
votes
3answers
328 views

Unbalanced target variable in Orange, how do I balance it?

So I want to perform a predictive model to predict churn. I have 2 datasets, one with churn and the other without (so I can later perform predictions). The issue is that I think my Confusion matrix ...
0
votes
0answers
106 views

XGBoost multiclass class balancing using weight parameter [duplicate]

I have three classes in the target variable with representation ratios of, class A:0.5 class B:0.3 ...
2
votes
2answers
46 views

oversampling data with subclass

Oversampling of under-represented data is a way to combat class imbalance. For example, if we have a training data set with 100 data points of class A and 1000 data points of class B, we can over ...
1
vote
0answers
49 views

Kappa Goes up as Accuracy Goes Down

I have recently been trying to train a randomForest model on a binary outcome with a very uneven class split. 282 control ~82% 63 case ~18% There are a total of 147 predictors that I'm testing for ...
6
votes
1answer
193 views

How to compare two unsupervised anomaly detection algorithms on the same data-set?

I want to solve an anomaly detection problem on an unlabeled data-set. The only information about this problem is that the anomalies population is lower than 0.1%. It should be notice that the size of ...
1
vote
1answer
50 views

What are some possible reasons that your multiclass classifier is classifying alll the classes in a single class?

I have unbalanced classes. Group1 N = 140 Group2 N = 35 Group3 N = 30 I ran the code on this data and all the Groups got classified as Group1. I thought that since group1 is the majority group this ...
4
votes
1answer
381 views

Overfitting - how to detect it and reduce it?

I have a side project where I am doing credit scoring using R (sample size around 16k for train data and 4k for test data, and also another two 20k data batches for out-of-time validation) with ...