1
$\begingroup$

I have performed 10 fold cross validation for performance estimation on a keras model. I now want to train a final neural network using ALL data for the FINAL model that will be deployed.

Given that this is the recommendation (to generate final model on all data), and I now no longer have any holdout data, conceptually, could I also average epochs taken to train during cross validation over the 10 folds to determine an epoch range that will not result in overfitting? or should i simply run 'x' epochs, overshoot and then simply determine the plateu?

is there a general rule here?

or... should I still have a validation split during training but no test set. This option makes no sense to me as this is exactly what cross validation was for.

thanks!

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Does this answer your question? New parameters in final training $\endgroup$
    – 10xAI
    Jul 14 '20 at 7:43
  • $\begingroup$ its a really nice thread but unfortunately not! this only deepens the debate about whether I should even train a final model after cross_val...they make great points about there being no metric by which you could test under or overfitting when you train on all the data.. and this is the issue i am having, i am not sure how to tackle this? -- the thread you provide seems to suggest to save the model for one of the folds in CV or re-train with some holdout set.... $\endgroup$
    – user9317212
    Jul 14 '20 at 8:26
0
$\begingroup$

If you use the scikit-learn GridSearchCV class (from sklearn.model_selection) together with the scikit-learn wrapper in keras, you can get your final model refit on the whole training set directly via the best_estimator_ attribute (i.e. the model instanced with the best hyperparms found in the CV process) already refit with the whole training dataset. I have seen some source which make a "manual" retrain on the whole training set, but it is not necessary as scikit-learn already let's you accessing the refit model on the whole set. Interesting links:

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for this!! So although this is refitting the whole model on the CV data (I used the kfold function and then trained in a loop for 10x Val) , it seems to indicate this is just effectively looping over different hyperparameters (epochs etc) .. I guess then ultimately if one were to do this manually, a keras model would be fit on all data with no validation or test data whatsoever? The keras scikit wrapper page suggests the same thing? $\endgroup$
    – user9317212
    Jul 13 '20 at 21:54

Not the answer you're looking for? Browse other questions tagged or ask your own question.