My Model is defined as below:
## defining the model
batch_size = 1
def my_model():
input_x = Input(batch_shape=(batch_size, look_back, 4), name='input')
drop = Dropout(0.5)
lstm_1 = LSTM(100, return_sequences=True, batch_input_shape=(batch_size, look_back, 4), name='3dLSTM', stateful=True)(input_x)
lstm_1_drop = drop(lstm_1)
lstm_2 = LSTM(100, batch_input_shape=(batch_size, look_back, 4), name='2dLSTM', stateful=True)(lstm_1_drop)
lstm_2_drop = drop(lstm_2)
y1 = Dense(1, activation='relu', name='op1')(lstm_2_drop)
y2 = Dense(1, activation='relu', name='op2')(lstm_2_drop)
model = Model(inputs=input_x, outputs=[y1,y2])
model.compile(loss='mse', optimizer='adam',metrics=['mse'])
model.summary()
return model
model = my_model()
It is a Stateful LSTM model with batch size =1. My model.fit
looks like this :
# Train the model
history = model.fit(
x_train,
[y_11_train,y_22_train],
epochs=1,
batch_size=batch_size,
verbose=0,
shuffle=False)
model.reset_states()
My model runs well and outputs results. But I am unable to increase or alter the batch_size
as flexibly as we could do when the model is in stateless condition.
As in for bigger size of dataset the model seems to be training forever as the batch_size
here is just 1. And as we know we can't randomly put any batch_size
value for stateful LSTM
as it needs to be a divisible factor.
I have gone through some blogs which describes changing batch_size
by using different batch sizes for training and predicting using the get_weights()
and set_weights()
functions in the Keras API ref: https://machinelearningmastery.com/use-different-batch-sizes-training-predicting-python-keras/, still it seems to be that the batch_size
that is being used here is less.
My question is : Cant we use batch size like 32, 64, 128 in stateful LSTMs? If yes then how do I implement it in my above given model, if not then what are the alternatives?
Looking for valuable suggestions.
Post Edit
In stateful LSTM model.reset_states()
should be after every epoch, Hence I set the resetting of the states after each epoch in the following way:
for i in range(100):
start = time.time()
history = model.fit(x_train, [y_11_train,y_22_train], epochs=1, batch_size=batch_size, verbose=0, shuffle=False)
model.reset_states()
print("Epoch",i, time.time()-start,"s")