2
$\begingroup$

I am training a neural network and doing 10-fold cross validation to measure performance. I have read lots of documentation and forums telling that the set of weights that should be saved or checkpointed are the ones that results to lowest val_loss and not highest val_accuracy, since the former usually results to higher testing accuracy.

Out of curiosity, I checkpointed both highest val_accuracy and lowest val_loss during my training. However, I found out that for some folds, I am getting better testing accuracy when I use the set of weights with highest val_accuracy compared to lowest val_loss. So during my cross-validation, I chose the set of weights that resulted to higher testing accuracy regardless of whether it came from the highest val_accuracy or lowest val_loss, and then just averaged the resulting testing accuracy across the 10 folds.

Is my methodology valid?

$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.