Can we use both cross validation/nested cross validation technique and early stopping with patient at the same time? Using early stopping for each (training, validation) fold and get best result of each (training, validation) fold and finally get average result as usual?

I have read an article about this situation Machine Learning mastery But it is not convincing enough for me.

  • $\begingroup$ If you're doing nested CV and your inner loop is doing some type of hyper parameter optimization you cannot take the averages of those results and report them. The inner loop would be used to find the best model and then that model would be retrained and tested on a held out dataset. If I remember correctly. $\endgroup$
    – IsmailE
    Mar 29 at 18:20
  • $\begingroup$ @IsmailE Thanks. But my primary question is about using "cross validation/nested cross" validation with "early stopping" not only cross validation/nested cross validation for tuning hyper parameters. $\endgroup$
    – Hoang Le
    Mar 29 at 18:24
  • $\begingroup$ why not just make the number of epochs also a hyper parameter if you're worried about over fitting. This I think would give you better insight into your problem. Depending on how diverse your dataset is I don't see a scenario where for each split the model wouldn't converge at around the same epoch. Also if your changing other parameters and then applying early stopping you won't be able to really gauge how well the other parameters are performing. $\endgroup$
    – IsmailE
    Mar 29 at 18:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.