0
$\begingroup$

Is hyperparameter tuning done on training or validation data set? The post here gives mixed opinion as of whether the training set should be used for hyperparameter tuning. And I would like to know whether hyperparameter tuning can be done on training data set?

More, I want to know what are the consequences for why we should/should not hyperparameter-tune on training dataset.

Thanks in advance!

$\endgroup$
1
  • $\begingroup$ Hyperparameter tuning involves both training and validation dataset, because you need a fitted model prior to the evaluation on a validation set. The validation set tells you which model to choose. Metrics computed on training set are NOT to be used on hyperparameter tuning, metrics on the validation set are. $\endgroup$
    – Ciodar
    Commented May 8, 2023 at 7:08

1 Answer 1

1
$\begingroup$

Tuning hyperparameters on the training set is generally not as critical as training on the test set, but overall increases the risk of overfitting.

Alternatively, use cross-validation to tune the hyperparameters. You might even get less biased hyperparameters, at the cost of higher implementation and computational demand (you have to train your model multiple times).

$\endgroup$
2
  • $\begingroup$ "training on the test set"? $\endgroup$
    – lpounng
    Commented May 10, 2023 at 7:43
  • $\begingroup$ @Ipounng Did you read what I wrote? "Training on the test set" is a critical problem. "Tuning on the train set" is less serious, yet still a (potential) problem. $\endgroup$ Commented May 10, 2023 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.