0
$\begingroup$

I have a hard time understanding the LSTM model performance as I summarize my model as follow:

X_train.shape
(120, 7, 11)
y_train.shape
(120,)
X_test.shape
(16, 7, 11)
y_test.shape
 (16,)

model = keras.Sequential()
model.add(keras.layers.LSTM(100, input_shape=(X_train.shape[1], X_train.shape[2]), return_sequences = True))
model.add(keras.layers.Dropout(rate = 0.2))
model.add(keras.layers.LSTM(20))
model.add(keras.layers.Dropout(rate = 0.2))          
model.add(keras.layers.Dense(1))
model.compile(loss='mean_squared_error', optimizer=keras.optimizers.Adam(0.001), metrics = ['mae'])

history = model.fit(
X_train, y_train, 
epochs=60,
batch_size=5,
verbose= 0,
validation_split = 0.1,
shuffle=False
)

Based on the below plots, both MSE and MAE decrease in the training process and their corresponding values are near zero.

MSE mae

However the prediction is not precise enough as I realize:

y_pred = model.predict(X_test)
model.evaluate(X_test,y_test)
[0.04673878103494644, 0.15574690699577332]

prediction

So my question is, how does really my model perform? I mean how can interpret its performance,since both MSE and MAE seem to be low but the prediction values are not quite convincing.

$\endgroup$
0
$\begingroup$

You are getting loss near to 0 but, Your true distribution of y in the range of 0-1 so, that 0.04 loss may be high loss. Just get random model and check the loss. You will get to know how much you decreased the loss. I will suggest to use r^2metric for evaluation.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.