My Model is defined as below:
## defining the model batch_size = 1 def my_model(): input_x = Input(batch_shape=(batch_size, look_back, 4), name='input') drop = Dropout(0.5) lstm_1 = LSTM(100, return_sequences=True, batch_input_shape=(batch_size, look_back, 4), name='3dLSTM', stateful=True)(input_x) lstm_1_drop = drop(lstm_1) lstm_2 = LSTM(100, batch_input_shape=(batch_size, look_back, 4), name='2dLSTM', stateful=True)(lstm_1_drop) lstm_2_drop = drop(lstm_2) y1 = Dense(1, activation='relu', name='op1')(lstm_2_drop) y2 = Dense(1, activation='relu', name='op2')(lstm_2_drop) model = Model(inputs=input_x, outputs=[y1,y2]) model.compile(loss='mse', optimizer='adam',metrics=['mse']) model.summary() return model model = my_model()
It is a Stateful LSTM model with batch size =1. My
model.fit looks like this :
# Train the model history = model.fit( x_train, [y_11_train,y_22_train], epochs=1, batch_size=batch_size, verbose=0, shuffle=False) model.reset_states()
My model runs well and outputs results. But I am unable to increase or alter the
batch_size as flexibly as we could do when the model is in stateless condition.
As in for bigger size of dataset the model seems to be training forever as the
batch_size here is just 1. And as we know we can't randomly put any
batch_size value for
stateful LSTM as it needs to be a divisible factor.
I have gone through some blogs which describes changing
batch_size by using different batch sizes for training and predicting using the
set_weights() functions in the Keras API ref: https://machinelearningmastery.com/use-different-batch-sizes-training-predicting-python-keras/, still it seems to be that the
batch_size that is being used here is less.
My question is : Cant we use batch size like 32, 64, 128 in stateful LSTMs? If yes then how do I implement it in my above given model, if not then what are the alternatives?
Looking for valuable suggestions.
In stateful LSTM
model.reset_states() should be after every epoch, Hence I set the resetting of the states after each epoch in the following way:
for i in range(100): start = time.time() history = model.fit(x_train, [y_11_train,y_22_train], epochs=1, batch_size=batch_size, verbose=0, shuffle=False) model.reset_states() print("Epoch",i, time.time()-start,"s")