I am getting better results with under sampling compared to weight class modification? what could be the possible reason?
2
-
1$\begingroup$ How much better are we talking, and in terms of what measure(s) of performance? $\endgroup$– DaveCommented Aug 23 at 21:18
-
1$\begingroup$ Are you sampling/weighting to the same effective positive class ratio? You're not downsampling the test set too, are you? (And I second Dave's request for which metric, and its values, ideally on a repeated split so we could evaluate whether the effect is just random.) $\endgroup$– Ben Reiniger ♦Commented Aug 29 at 21:56
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
There could be many reasons that you are achieving better results with undersampling compared to changing the class weights for your machine learning model.
Even though undersampling theoretically leads to data loss and may reduce the predictive performance of a model, this may not always be specifically the case. Undersampling, in my previous experience, has a few times led to a better model predictive performance then SMOTE or other resampling techniques, though.
To answer your question about the reason for this, it could be that for your specific dataset, it has underlying patterns that just make undersampling make the predictive performance better than adjusting the class weights. Without knowing more about what your dataset is and the specifics of it, it is hard to say the exact reason for this.
It really depends on the dataset to see which resampling technique does the best. It is best to really try all of them with K-fold cross validation (or do hyperparameter tuning) to see which one gets the best predictive performance.
