I have a dataset separated in train, test and validation splits.

After each epoch, I evaluate the loss and accuracy in the validation split.

When the loss in validation split is not better, I stop train and choose that as final model.

But, I should merge train and validation as final model? How can I choose the best model?

  • $\begingroup$ Welcome to the site! what you are talking is the right procedure. So, during generating a model you have to use validation set to choose the right model(model which ever you feel is right for your analysis). Once the model is selected you can train the model with the whole data in the hope of making the model better to perform better on your actual data. $\endgroup$ – Toros91 Aug 27 '19 at 1:59
  • $\begingroup$ Thank's you Toros91. I really appreciate your help. Is the number of epochs "part of the model"? Hence, i should select a specific number of epoch in this step? $\endgroup$ – Xtalker Aug 27 '19 at 2:13
  • 1
    $\begingroup$ Yes, the number of epochs are important. You have to do some trail and error to find which number of epochs gives you better and accurate results. $\endgroup$ – Toros91 Aug 27 '19 at 2:17

I really like this post with the answer on your question: https://machinelearningmastery.com/train-final-machine-learning-model/

Once we have the estimated skill, we are finished with the resampling method.

You finalize a model by applying the chosen machine learning procedure on all of your data.

It means that after achieving enough performance of the model, we don't need anymore splitting ("resampling methods"). Then we can merge all available data and repeat learning procedure. We can rely on the final performance (have no qualms about, who model will behave using new data during training), because it has already learned how to generalized input for the particular task (differ cats from dogs for example).

The careful design of your test harness is so absolutely critical in applied machine learning. A more robust test harness will allow you to lean on the estimated performance all the more.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.