Timeline for Oversampling only balances the training set, what about the testing set?
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 21, 2019 at 6:21 | comment | added | Simon Larsson | Are you saying that you still end up with no positive samples in your test set? | |
| Aug 21, 2019 at 6:20 | comment | added | Simon Larsson | Yes. Let's say I have a dataset with 5 positive samples (y=1) and 1 million negative samples (y=0). If I make 5 stratified folds, then I will end up with 1 positive sample per fold. Each train set will be 4 of those folds and have 4 positive samples. Each test set will be on 1 fold and have 1 positive sample. So the imbalance does not matter, you will end up with positive samples in your test set if you do stratified folds. | |
| Aug 21, 2019 at 4:05 | comment | added | Sarah | Are you sure about this part of your answer? "... It will distribute your classes evenly across all folds." I thought stratifying preserves the distribution of the original data. When I apply SMOTE on the training set (after splitting the data to train/test), train and test sets will have completely different distributions [my original dataset is HIGHLY imbalanced]. | |
| Aug 20, 2019 at 22:45 | comment | added | Sarah | Unfortunately, stratifying did not do the trick! I am going to implement the hierarchical idea and will let you know if it helps! | |
| Aug 20, 2019 at 20:18 | comment | added | Simon Larsson | It is worth trying if just stratifying does not do the trick. | |
| Aug 20, 2019 at 20:13 | vote | accept | Sarah | ||
| Aug 20, 2019 at 19:58 | comment | added | Sarah | Thank you! I was thinking about using a Hierarchical Classification (or a step-wise classification) and exclude the majority class in step #1, the second majority class in step #2 and so on. I have a 5-class classification problem. However, I am going to try StratifiedKFold to see how it works. Any thoughts on the Hierarchical idea? | |
| Aug 20, 2019 at 19:51 | history | answered | Simon Larsson | CC BY-SA 4.0 |