1
$\begingroup$

Let's assume I have 2 models

Model 1:

  • Train Accuracy = 92.4%
  • Validation Accuracy = 37.6%
  • Test Accuracy = 35.3%

Model 2:

  • Train Accuracy = 37.0%
  • Validation Accuracy = 34.2%
  • Test Accuracy = 34.1%

Which is the best model ? Model 1 is heavily overfitting but the final performance is better

Improve this question
$\endgroup$
1
  • $\begingroup$ Oof, I forgot that I can't just propose a duplicate now. Let me know if the linked question doesn't help, and I'll reopen. $\endgroup$
    – Ben Reiniger ♦
    Sep 7, 2022 at 13:35

1 Answer 1

Reset to default
0
$\begingroup$

Deep learning models heavily rely on stochastic processes such as weight initialization, back-propagation, etc. For evaluation and comparison of different models, there are methods that are generally referred to as Cross-Validation. The most popular type of CV is the k-Fold CV, and if your model training comprises hyperparameter tuning, you must use the Nested k-Fold CV.

Improve this answer
$\endgroup$
0

Not the answer you're looking for? Browse other questions tagged or ask your own question.