I am training a keras model that utilizes
early_stopping in order to prevent overfitting. This requires that I set aside a validation dataset.
My task requires that I keep my training and validation split by time, so that all samples in my validation set occur after the point in time of those in my training set.
My challenge is that the examples in my validation (by definition the most recent examples in time) are very important for my prediction task and I would like to use them to train a final model. From all I can see, it seems that in general it is recommended to train a final model (to be released to production) on all data available, after model configuration has been decided upon in a traditional train/test period (see here).
However, if I use all of my data to train a final model, I no longer can utilize
early_stopping, since I will not have any validation set (it will be being used for training).
I could randomly sample a subset of my training data to use for validation (instead of using the most recent data as I was during training/testing), but then I worry that due to the time series dynamic of the problem I am running the risk of data leakage.
My question really boils down to:
What is the preferred way to train a final, production model when in Keras (or another framework)?