I have a training and a test dataset. I would like to use the output of Model A in an ensemble model. However, I would like to use early stopping.

Usually, I would create Model A for each K-fold (on training) and predict the OOF to create the meta-model dataset. Then I would repeat the methodology but for hyperparameter tuning the 2nd layer of the stacking model, trained with the meta-model dataset.

Finally, I train the meta models on the whole training set, train the 2nd layer of the stacking model on the whole training set, and predict\evaluate the results for the test set.

How would the final step work if I am using early-stopping? should I just create the meta-model dataset without CV and without predicting the OOF? or should I forgo early stopping and find the right hyper-parameters\training steps.

  • $\begingroup$ what does OOF stand for? $\endgroup$
    – Ben
    Aug 3 '20 at 11:40
  • $\begingroup$ out of fold.... $\endgroup$
    – targetXING
    Aug 3 '20 at 19:55
  • 1
    $\begingroup$ Early stopping is used when your model stops learning or in other words accuracy does not increase. $\endgroup$
    – Syenix
    Aug 4 '20 at 4:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.