0
$\begingroup$

Does anyone know, if it is ok if I use early callbacks with restore best weights? The metric measured by the early callback is validation loss. I was afraid that if I restore the best weights of the NN when the val_loss is minimum, my model will somehow basically learn on the validation dataset. Is this a good practice?

$\endgroup$

4 Answers 4

0
$\begingroup$

take a look here: https://medium.com/@doleron/never-use-restore-best-weights-true-with-earlystopping-754ba5f9b0c6 it is not so good to restore best weights.

$\endgroup$
0
$\begingroup$

This seems fine to me. Yes, you are using your validation dataset in the training process, but that is no different from using your validation dataset in other parts of the training process, such as hyper-parameter tuning or early stopping. This is one of the reasons to use a validation dataset and so isn't a problem - assuming you also have a separate test dataset, which you only use to evaluate the final model.

$\endgroup$
0
$\begingroup$

This is exactly what validation does. You do not panic when choosing a model with the best val_loss that your choice affects the performance of the model, right? Stopping early just does what you should have done earlier on your own.

$\endgroup$
0
$\begingroup$

Yes, it is a common and good practice to use early stopping callbacks with restore best weights. This approach helps to prevent overfitting on the training data.

When you use early stopping, the training process is halted once the model's performance on the validation set stops improving for a specified number of epochs (the "patience"). This helps to ensure that the model doesn't overfit to the training data by continuing to learn noise or non-generalizable patterns.

The "restore best weights" option means that the model's weights are rolled back to the state where it had the best performance on the validation set. This is typically what you want, as these weights have so far provided the best generalization performance.

This doesn't mean that your model is learning from the validation set in the same way it learns from the training set. The validation set is only used to monitor the model's performance on unseen data and to make decisions about when to stop training and which weights to keep. The model's parameters are not updated based on the validation set during the training process.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.