Questions tagged [imbalanced-data]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
1answer
32 views

Improving text classification & labeling in imbalanced dataset

I am trying to classify text titles (NLP) in categories. Let us say I have 6K titles that should fall into four categories. My questions: I do not understand why in some ML techniques categories are ...
2
votes
2answers
73 views

Determining if a dataset is balanced

I'm learning about training sets and I have been provided with a set of labelled customer data that segments customers into one of two classes: A or B. The dataset also contains gender, age and ...
0
votes
1answer
16 views

Aren't balanced data sets important in regression?

Why is it that the necessity for balanced data sets is (almost) always exclusively mentioned in the context of classification but not of regression?
1
vote
1answer
41 views

How to increase the accuracy of an imbalanced dataset (not precision)?

There's an imbalanced dataset in a Kaggle competition I'm trying. The target variable of the dataset is binary and it is biased towards 0. 0 - 70% 1 - 30% I tried several machine learning algorithms ...
0
votes
1answer
24 views

My semantic segmentation model classifies everything as background

So, I am working on a semantic segmentation task using U-Net. The dataset is very unbalanced, with the background being by far the most common, and the last class being very scarce. First I trained it ...
0
votes
1answer
38 views

Why is the f1 score of my imbalanced data for a multiclass problem so low?

I am dealing with a multi-class problem and imbalanced data. I am trying to find an algorithm that can predict well each class with python (sklearn and pandas). My dataset contains: 620 rows, 12 ...
0
votes
0answers
28 views

What do you suggest to increase the sensitivity rates in conventional ML model?

I have an imbalanced data problem (prop. rate: 0.8571429 0.1428571) and for this reason, our sensitivity and PPV rates are very low. What do you recommend to fix this problem in R or in general? See ...
0
votes
0answers
20 views

How to apply class weights for imbalanced data and given misclassification costs?

Say I have a dataset for binary classification with 1000 samples in the minority class and 100000 samples in the majority class and use XGBoost to output probabilities. From an expert I know that ...
0
votes
1answer
55 views

What is the best practice to normalize/standardize imbalanced data for outlier detection or binary classification task?

I'm researching Anomaly/outlier/fraud detection, and I'm looking for the best practice to pre-process the synthetic data for imbalanced data. I have checked all methodology for normalizing/...
2
votes
1answer
29 views

On what threshold we should resample the data?

When working with churn datasets, we usually find imbalanced datasets. My question is how to decide on what basis we should resample the data. For example: while splitting the data before training we ...
2
votes
0answers
38 views

Is balancing class data for imbalanced problems helpful or just folklore when considering thresholds?

Caveat: I'm aware that imbalanced data questions are a dead horse, but I haven't found an answer to this flavor of it directly. When working with highly imbalanced data (e.g. binary class cases), ...
1
vote
1answer
33 views

Can the attention mask hold values between 0 and 1?

I am new to attention-based models and wanted to understand more about the attention mask in NLP models. attention_mask: an optional torch.LongTensor of shape [...
1
vote
1answer
31 views

Dealing with high frequency tokens during masked Language modelling?

Suppose I am working with a Masked Language Model to pre-train on a specific dataset. In that dataset, most sequences have a particular token of a high frequency ...
2
votes
2answers
47 views

Doesn't over(/under)sampling an imbalanced dataset cause issues?

I'm reading a lot about how to use different metrics specifically for imbalanced datasets (e.g. two classes present, but 80% of the data is one class) and how to tackle the issue of imbalanced ...
0
votes
2answers
40 views

Do we have any method which handles the imbalanced classification with sample weighting instead of class weighting?

I am looking for methods that use sample weighting instead of class weighting for the imbalanced classification. I think sample weighting is more precise than weighting all the samples from one class ...
0
votes
0answers
18 views

How to tune the proportion of minority class after oversampling in imbalanced data with cross-validation? [duplicate]

I have an imbalanced data set and I want to balance them with SMOTENC with cross-validation. In order to determine the performance of the classifier on the original data, I will cross-validation as ...
0
votes
0answers
46 views

How to apply variational autoencoder for oversampling with cross-validation?

Currently, I have an imbalanced data set with proportions 84% and 16%. I wanna use VAE as oversampling method and I want to determine the best proportions of data that results in better metrics. Also, ...
8
votes
1answer
269 views

Categorization of approaches to deal with imbalanced classes

What is the best way to categorize the approaches which have been developed to deal with imbalance class problem? This article categorizes them into: Preprocessing: includes oversampling, ...