I am training a simple LSTM network using Keras to predict time series values. It is a simple 2-layer LSTM.
I get the best performance when I train on subsets of the training set that start at random points. Each subset has a training size of 100 samples and a validation size of 30 samples. At each sample the model has a batch size of 16, trains for 100 epochs with an early stop after 20 epochs of little improvement. I run this training 10 times.
How is this different from me training me simply training my model as such:
model.compile(loss='mean_squared_error', optimizer='adam') results = model.fit(X_train, y_train, epochs=100, batch_size=32, shuffle=False, validation_split=0.3, verbose = 1, steps_per_epoch=10, callbacks = [EarlyStopping(monitor='val_loss', min_delta=5e-5, patience=20, verbose=1)])
Doesn't Keras automatically select random subsets of the training data for training and validation through the mini batches? Is it simply because I am training the model so many more times?
Is this effectively a k-fold-esque training implementation? However, using sk-learn's KFold or Repeated Kfold doesn't get results as good.
NB: I do not want to use shuffle as this is time series data and that would distort the model training.