Error with MSE in LSTM

Question

I'm trying to fit an LSTM model on my dataset, using also a validation set. My datasets have the following shapes:

X_train = (56054, 250, 30) #where 250 = sequence_length
X_val = (13969, 250, 30) #where 250 = sequence_length

This is the model I created:

cbs = [History(), EarlyStopping(monitor='val_loss', 
    patience=3, min_delta=0.0003, verbose=0), 
    TensorBoard(log_dir='Baseline/tb_logsLSTM', histogram_freq=1, write_images=True)]

model = Sequential()

model.add(LSTM(40, input_shape=(None, X0train_seq.shape[2]),
    return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(40, input_shape=(None, X0train_seq.shape[2]),
    return_sequences=False))
model.add(Dropout(0.2))
model.add(Dense(30))
model.add(Activation('linear'))
model.compile(loss='mse',
    optimizer='adam', metrics=["mse"])
model.fit(X_train, X_train,
    batch_size=60,
    epochs=35,
    validation_data=(X_val, X_val),
    callbacks=cbs, verbose=True)

When I run it, it finish the first epoch and give me this error in the fit function:

tensorflow.python.framework.errors_impl.InvalidArgumentError:  Incompatible shapes: [64,30] vs. [64,250,30]
 [[node gradient_tape/mean_squared_error/BroadcastGradientArgs

How can I solve it?

Answer 1

It seems a couple of things can be done differently.

Firstly, it seems you are passing train and label data incorrectly when fitting the model. It should be more like:

model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(trainX, trainY, epochs=100, batch_size=1, verbose=2)

For trainY being your labels, as opposed to passing trainX twice as in your code above. Same applies for your validation_data argument.

Secondly, what is the role of a linear activation in your code? You could simply close your computational graph with the dense layer e.g.

model = Sequential()
model.add(LSTM(4, input_shape=(1, X0train_seq.shape[2])))
model.add(Dense(1))
model.compile(...

Hope this is helpful.