0
$\begingroup$

Im trying to use gridsearch to find the best parameter for my model. Knowing that I have to implement nearmiss undersampling method while doing cross validation, should I fit my gridsearch on my undersampled dataset (no matter which under sampling techniques) or on my entire training data (whole dataset) before using cross validation?

$\endgroup$
3

2 Answers 2

0
$\begingroup$

Do grid search on the same Level of "imbalancedeness" that you plan/are able to do your Training and Evaluation on.

So that means that if you saw that imbalanced data set does not skew your model predictions or results in other unwanted Outcomes, done use the maximal dataset possible. But on the other Hand if your model is strongly overfitting because of imbalanced dataset then optimisation with grid search will make him overfit more in that direction.

$\endgroup$
1
  • $\begingroup$ Thank you very much for your answer ! So if my dataset is imbalance, there is no point using grid search on a resampled dataset right ? $\endgroup$
    – Valentin
    Feb 16, 2021 at 15:28
0
$\begingroup$

You can create the grid search - cross validation manually instead of using GridsSearchCV and for each split upsample the rarest class only for the folds used for training. The fold used for validation stays the same. This because if you upsample all the training set before the cross validation, you will have a too optimistic result for the validation error. In both the training portion and validation portion of the split there would be the same duplicated instances (and possibly plenty of them)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.