1
$\begingroup$

I'm unclear on the exact process of using the validation data.

Let's say that I fit my neural network model and adjust hyperparameters using the training set and validation set. Do I then evaluate the test set on this model? Or do I recombine the validation and training sets and fit a fresh model with the hyperparameters that I found during the validation phase, and then evaluate on the test data? I have seen a number of different notebooks and examples that do both ways.

Surely, once I've found my hyperparameters, it makes sense to fit a fresh model using the full training set (recombined with validation set), since the validation loss has no effect on the weights.

$\endgroup$
3
$\begingroup$

The common procedure is this:

Fit the model on train split and adjust hyperparameters using the training split and validation split, as you already said.

Then, to get a "final measure" of performance, train model on both training and validation splits using the hyperparams found during validation phase, and evaluate on test split.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.