1
$\begingroup$

I'm using GridSearchCV to tune hyperparameters for a Logistic Regression multiclass model.

I read on Kaggle that you should choose the hyperparameter that results in the lowest discrepancy between the CV-score and the training score, but in this case this leads to a very low score.

How should I choose the proper C value to ensure generalisability of the model but also high model performance based on the CV-curve below?

enter image description here

From my understanding opting for low discrepency between the two scores ensures the ability of the model to be generalised to unseen data. But on the other hand I want a score as high as possible on unseen data.

Thanks for any help!

$\endgroup$
1
$\begingroup$

Choosing the best validation accuracy is the common practice, since validation is unseen data.

Sometimes you might have over-fitting to the validation set, mainly if it is too small or no very representative of the data (for example if it has considerably more examples of one class, thus a good model would be a model that says that (almost) everything belongs to that class).

If you are worried about over-fitting, you could increase your regularization strength.

$\endgroup$
  • $\begingroup$ I do have a small validation set, that's not completely balanced, Thank you for your answer. I'm going for a bit lower C value. $\endgroup$ – DataFace Oct 24 '19 at 18:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.