Questions tagged [imbalanced-learn]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
3answers
22 views

Problem of having imbalanced classes in the test set while using oversampling

I have an imbalanced dataset. My classes are 0 and 1. The number of 0 class instances is about 20 times more than the number 1 class instances. I know that I should apply oversampling after train test ...
2
votes
0answers
30 views

Using SMOTENC in a pipeline

I am trying to figure out the appropriate way to build a pipeline to train a model which includes using the SMOTENC algorithm: Given that the N-Nearest Neighbors algorithm and Euclidian distance are ...
1
vote
1answer
35 views

Why to adjust class weights instead of simply finding the best threshold?

In a binary supervised classification where classes 1 and 0 have different number of samples in training, it’s very common to find tutorials about adjusting class weights, over and under sampling for ...
1
vote
1answer
42 views

Cross validation schema for imbalanced dataset

Based on a previous post, I understand the need to ensure that the validation folds during the CV process have the same imbalanced distribution as the original dataset when training a binary ...
2
votes
2answers
72 views

For imbalanced classification, should the validation dataset be balanced?

I am building a binary classification model for imbalanced data (e.g., 90% Pos class vs 10% Neg Class). I already balanced my training dataset to reflect a a 50/50 class split, while my holdout (...
0
votes
1answer
13 views

Low leves of probability observed after modelling.Is it right to scale the probability

I have done modelling on imbalanced class , without any sampling methods. Event rate is around 0.1 ,After modelling I am getting probalities more at the lower side close to zero.I have tried differnt ...
2
votes
0answers
29 views

The most informative curve for imbalance datasets

For the imbalanced datasets: Can we say the Precision-Recall curve is more informative, thus accurate, than ROC curve? Can we rely on F1-score to evaluate the skillfulness of the resulted model in ...
0
votes
0answers
19 views

dealing with imbalanced data for multi-class problem

Based on the experiments I run for a number of times, and the reading I did on imbalanced data for a multiclassification problem such as this paper, resampling techniques like ...
0
votes
0answers
21 views

Predict best score on unlabelled test set

Data I have one dataset with $1500$ data points, each with $\sim 23 000$ features (gene expression data, if that matters). However, I've split this dataset into a labelled training set of size 1000, ...
1
vote
0answers
17 views

Multi-classification: low precision due to imbalanced classes in test data - what to do?

I built a multi-classification model with 3 result classes (XGBoost using R's caret-package): A, B and C. I undersampled my training data - so every class is equally abundant for training. The ...
3
votes
1answer
67 views

Why is oversampling outperforming class weight?

I have a dataset that is highly imbalanced. One class has 412 (class 0) samples while the other has 67215 (class 1) samples. For its classification, I am using MLP. When I use class weight of 165 for ...
0
votes
1answer
60 views

Preferred approaches for imbalanced data

I am building a binary classification model with imbalanced target variable (13% Class 1 vs 87% class 0). I am considering the following three options to handle the data imbalance Option1: Create a ...
1
vote
0answers
27 views

CART classification for imbalanced datasets with R

Hey guys i need your help for a university project. The main Task is to analyze the effects of over/under-smapling on a imbalanced Dataset. But before we can even start with that, our task sheet says, ...
3
votes
0answers
88 views

Balancing the dataset using imblearn undersampling, oversampling and combine?

I have the imbalanced dataset data['Class'].value_counts() Out[22]: 0 137757 1 4905 Name: Class, dtype: int64 ...
1
vote
1answer
910 views

SMOTE for regression

I am looking into upsampling an imbalanced dataset for a regression problem (Numerical target variables) in python. I attached paper and R package that implement SMOTE for regression, can anyone ...
1
vote
1answer
29 views

Quantifying the imbalanceness of a dataset

after looking a lot in the literature there is really a lot of how to work with imbalanced dataset but so far I can not find a definition of a imbalanced metric that quantifies how much imbalanced ...
1
vote
0answers
29 views

Balance two crossentropy losses with different number of neurons

I have a model with a few outputs, each output with shape: Shape: (batch_size, labels_1) -> softmax -> ...
0
votes
0answers
8 views

How to increase Overlapping in Real Dataset

How to increase overlapping in real data set, i.e, if we add some sample in majority class or add sample in overall data set the overlapping may increase, but the question is that how to add sample in ...
0
votes
2answers
713 views

How to find whether a dataset is blanced or imbalanced?

I have few dataset to experiment classification(Multi-class). These datasets are about 400GB. I wanted to know whether the dataset is balanced or imbalanced. How to know that dataset is balance or ...
1
vote
3answers
2k views

How to Split And Resample Imbalanced Dataset Into Train, Validation and Test

I want to understand how to split the imbalanced data set with a binary target variable where 87% of the samples are negative and 13% of the samples are positive. Now, I know that you should always ...
0
votes
0answers
45 views

How to train regression models on imbalance dataset?

I am trying to train a regression model on the imbalance dataset. For example, the input features are vehicle_type, ...
0
votes
1answer
47 views

Evaluate imbalanced classification model on balanced testing sample

Why it would be too optimistic to compute presicion, recall and f1-score to evaluate a model trained for imbalanced classification on a balanced testing sample ?
0
votes
0answers
11 views

Shrink the training set during the learning process

Is there any way to change the size of the training set during the learning process? For example, let's say we have four classes (with their distribution): [A (90%), B (5%), C(2%), D(3%)]. Can we ...
1
vote
0answers
38 views

Binary classifier on imbalanced dataset yields weird PR curve

I have a dataset with ~6M points, 9 features and two classes. The minority class represents just under 2% of the data. The data is first divided into 100 batches and a different classifier is trained ...
2
votes
1answer
594 views

Difference between sklearn make_pipeline and imblearn make_pipeline

Can anybody please explain the difference between sklearn.pipeline.make_pipline and imblearn.pipeline.make_pipline.
6
votes
2answers
98 views

Can we specify the number of data generated(minority class) using SMOTE?

I am trying to improve classification of imbalanced dataset creditcard fraud using SMOTE imbalanced_learn. But, in this it generates the data to 50%, can we give a specific number for the data to be ...
0
votes
1answer
251 views

Positively skewed target label in regression

I have a dataset where the target label is positively skewed and produces a long tail, and currently I have a high residual on these values when experimenting with some linear, tree-based and neural-...
1
vote
0answers
90 views

SMOTE and oversampling with constraints

I'm trying to apply SMOTE to a dataset that has time-constraints. I have information about users visiting a website. For some features, there are time constraints, e.g having the first visit and the ...
1
vote
2answers
170 views

A robust metric in the presence of class imbalance

When evaluating the performance of a multiclass classification problem, on a highly imbalanced dataset, what is the most robust metric for this purpose? I read a paper that states: "Average ...
0
votes
1answer
200 views

Best metric in imbalanced classification for multi-label classification

My test data are imbalanced, i tried to use the precision or the gmean as metrics for a multi-label learning model, but both metrics are not very informative. Is there any way to use for example the ...
0
votes
2answers
77 views

Dealing with the test set of imbalanced data

I am working on a problem dealing with unbalanced data that has a very specific request. I would like to know the following: When I have an imbalanced dataset and I do train test split, the test ...
1
vote
1answer
672 views

Improving accuracy on highly imbalanced dataset

I need some suggestions to improve my model accuracy. The training data shape is : (166573, 14) It has all int and float columns. I have dropped claims_daysaway column as most of values are NaN and ...
1
vote
1answer
881 views

Multi class Imbalanced datasets under-sampling imblearn

I have an imbalanced dataset. I am looking to under-sample. Even though, the oversampling process takes less time, the model training takes a lot of time. I have taken a look at imbalanced-learn ...
0
votes
0answers
91 views

Deep learning(MLP) on multiclass classification. Model learns only one class

I am new to deep learning. I have imbalanced class data. I used one hot encoding and scaling to preprocess my data. I have used adamoptimizer as optimizer function and sparse categorical crossentropy ...
2
votes
3answers
1k views

imbalanced dataset in text classififaction

I have a data set collected from Facebook consists of 10 class, each class have 2500 posts, but when count number of unique words in each class, they has different count as shown in the figure Is ...
3
votes
1answer
620 views

How to use SMOTENC inside the Pipeline?

I would greatly appreciate if you could let me know how to use SMOTENC. I wrote: ...