All Questions
Tagged with class-imbalance python
72 questions
1
vote
0
answers
89
views
Is it right method to remove instances that are hard to predict before train test split?
In a binary classification problem,
I have a slightly unbalanced medical dataset with class distribution:
0:5600, 1:1500
0 without a problem and 1 with a problem.
I tried many pipelines, automls, and ...
- 113
1
vote
0
answers
237
views
Stratified K Fold Cross Validation in Orange: python script
I am using Orange to predict customer churn and compare different learners based on accuracy, F1, etc.
As my problem is unbalanced (10% churn - 90% not churn), I want to oversample. However, when ...
1
vote
0
answers
40
views
How to fix class imbalance in dialogue (text) time series data?
I have a dataset that looks like this:
...
- 133
1
vote
0
answers
425
views
Ensure class balanced batches while hyperparameter tuning keras models with grid search
Ensuring class balanced batches while training keras models is possible using fit_generator method. I used imblearn.keras.BalancedBatchGenerator for that and it works fine!
But I wanted to do that ...
- 251
1
vote
2
answers
115
views
Determining threshold in an area with very few samples of positive label
I have a binary classification task where I want to either keep or discard samples. I have about a million samples, and about 1% should be kept. I want to discard as much as possible, but discarding ...
- 11
1
vote
0
answers
58
views
Different result between Rapidminer and Python imblearn
I'm currently working on imbalanced classification problem. However i found different result between SMOTE in rapidminer and SMOTE in imblearn (python). rapidminer SMOTE give 15-20% improvement on ...
- 11
1
vote
2
answers
3k
views
Scripting code for class imbalance in Biolabs Orange
I'm trying to manipulate some data in Biolabs Orange, using the built in Python Script widget and information at Biolabs Orange tutorial on scripting.
However, I'm struggling with taking the results ...
- 131
0
votes
2
answers
527
views
What makes the validation set a good representative of the test set? [closed]
I am developing a classification model using an imbalanced dataset. I am trying to use different sampling techniques to improve the model performance.
For my baseline model, I defined an AdaBoost ...
- 447
0
votes
2
answers
84
views
Imbalanced class with same rows? [closed]
In my dataset i have 3 classes-> 0,1,2.
0(72k),1(13k)and 2(13K) in brackets are there count.
So whenever i try to predict them with any algorithm ,i observed that almost all the "2"'s are predicted ...
- 141
0
votes
1
answer
131
views
Machine Learning Binary Classification Model on a Small Tabular Imbalanced Dataset - Improving Performance
I have a dataset that is fairly small (15,000 rows), with 10 features for a model to learn from. It is not possible to increase the size of this dataset. I am using machine learning for binary ...
- 173
0
votes
1
answer
625
views
Logistic regression with unbalanced data, scoring based only on rare class
I have a dataset off app. 600.000 data points in which 0.2% (1.200 samples) is labelled as signifying a rare event. I want to use logistic regression to help me predict this rare event, but even when ...
- 15
0
votes
1
answer
84
views
Any pythonic way to auto determine imbalance class problem, specially in multiclass scenario?
A data is imbalanced if a target class proportions are unequal and typically, heavily biased. But, what is the exact measurement of this heavy bias?
Before applying imbalance techniques (SMOTE, ADASYN,...
- 101
0
votes
1
answer
209
views
Model accuracy: how to determine it?
I have some doubts regarding the approach to building a classifier such as Multinomial Naive Bayes or SVM. I will go through the steps to see if the approach is fine. I do have not a lot of experience ...
- 295
0
votes
1
answer
286
views
Undersampling improvement F1-score
Im doing a 2-class classification project for an imbalanced data set.
The imbalance is about 18%/82%. Im noticing a huge improvement in F1-score when I under-sample; from 16% without under-sampling to ...
- 31
0
votes
0
answers
36
views
Multiclass PyTorch neural net is stuck predicting 1 class, even with "simple" dataset
I'm trying to predict a class of some data, and am struggling.
So, to debug I created a simple test dataset, yet I am having the same issues. I've tried adding weighting, and lastly adding a column ...
- 1
0
votes
0
answers
97
views
How to handle imbalance in input variables?
Currently working on a finance dataset which has more than 20 input variables with high imbalance.
[Apparently, the target variable is also imbalanced (for this I am currently considering to handle it ...
- 1
0
votes
0
answers
181
views
Different training score but same test score when using pipeline
I have a problem that produce different training score when using pipeline and manual.
MANUAL :
...
- 136
0
votes
1
answer
152
views
Data simulation using make_classification in Python
I have a question about data simulation in Python. I deal with the classification of imbalanced data and want to test the effectiveness of different methods on simulated data. I have seen in various ...
- 21
0
votes
1
answer
28
views
Unbalanced training set from balanced data
I am looking to get an unbalanced training set with a given ratio of classA:classB from a dataset without regarding if it is balanced or not. The point is to analyze the influence of data imbalance on ...
- 1
0
votes
1
answer
2k
views
How does class_weight work in Decision Tree?
I am interested in Cost-Sensitive learning. And I am trying to understand how class_weight in DecisionTree works in terms of math. I read a lot of articles that ...
- 21
0
votes
2
answers
225
views
Why does downsampling leads classification to only predict one class?
I have a multi-class classification problem. It performs quite well but on the least represented classes it doesn't. Indeed, here is the distribution :
And here are the classification results of my ...
