0
$\begingroup$

Is hyperparameter tuning done on training or validation data set? The post here gives mixed opinion as of whether the training set should be used for hyperparameter tuning. And I would like to know whether hyperparameter tuning can be done on training data set?

More, I want to know what are the consequences for why we should/should not hyperparameter-tune on training dataset.

Thanks in advance!

Improve this question
$\endgroup$
1
  • $\begingroup$ Hyperparameter tuning involves both training and validation dataset, because you need a fitted model prior to the evaluation on a validation set. The validation set tells you which model to choose. Metrics computed on training set are NOT to be used on hyperparameter tuning, metrics on the validation set are. $\endgroup$
    – Ciodar
    May 8 at 7:08

1 Answer 1

Reset to default
1
$\begingroup$

Tuning hyperparameters on the training set is generally not as critical as training on the test set, but overall increases the risk of overfitting.

Alternatively, use cross-validation to tune the hyperparameters. You might even get less biased hyperparameters, at the cost of higher implementation and computational demand (you have to train your model multiple times).

Improve this answer
$\endgroup$
2
  • $\begingroup$ "training on the test set"? $\endgroup$
    – lpounng
    May 10 at 7:43
  • $\begingroup$ @Ipounng Did you read what I wrote? "Training on the test set" is a critical problem. "Tuning on the train set" is less serious, yet still a (potential) problem. $\endgroup$
    – Martin Benes
    May 10 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.