# Multivariate LSTM RMSE value is getting very high

I want to predict a time series with multiple variables. I am using Keras's LSTM class.

Here is my data set description :

I want to predict var1(t-1) and my X variables are var3(t-1) , var4(t-1) , var5(t-1) , var6(t-1) and var7(t-1).

And this is how my model configuration looks like :

{
"data": {

"columns": [
"var1(t-1)",
"var3(t-1)",
"var4(t-1)",
"var5(t-1)",
"var6(t-1)",
"var7(t-1)"
],
"sequence_length": 5,
"train_test_split": 0.80,
"normalise": false
},
"training": {
"epochs": 100,
"batch_size":10
},
"model": {
"loss": "mse",
"optimizer": "nadam",
"save_dir": "saved_models_multi",
"layers": [
{
"type": "lstm",
"neurons": 200,
"input_timesteps": 5,
"input_dim": 5,
"return_seq": true
},
{
"type":"batch_norm"

},
{
"type": "dropout",
"rate": 0.4
},

{
"type": "lstm",
"neurons": 200,
"return_seq": true
},
{
"type":"batch_norm"

},

{
"type": "dropout",
"rate": 0.4
},
{
"type": "dense",
"neurons": 50,
"activation": "sigmoid"
},

{
"type":"batch_norm"

},

{
"type": "dropout",
"rate": 0.2
},

{
"type": "lstm",
"neurons": 200,
"return_seq": false
},
{
"type":"batch_norm"

},

{
"type": "dropout",
"rate": 0.4
},
{
"type": "dense",
"neurons": 25,
"activation": "sigmoid"
},

{
"type":"batch_norm"

},

{
"type": "dropout",
"rate": 0.2
},

{
"type": "dense",
"neurons": 1,
"activation": "linear"
}
]
}
}


This is number of parameters in each layer and output shape :

Layer (type)                 Output Shape              Param #
=================================================================
lstm_1 (LSTM)                (None, 5, 200)            164800
_________________________________________________________________
batch_normalization_1 (Batch (None, 5, 200)            800
_________________________________________________________________
dropout_1 (Dropout)          (None, 5, 200)            0
_________________________________________________________________
lstm_2 (LSTM)                (None, 5, 200)            320800
_________________________________________________________________
batch_normalization_2 (Batch (None, 5, 200)            800
_________________________________________________________________
dropout_2 (Dropout)          (None, 5, 200)            0
_________________________________________________________________
dense_1 (Dense)              (None, 5, 50)             10050
_________________________________________________________________
batch_normalization_3 (Batch (None, 5, 50)             200
_________________________________________________________________
dropout_3 (Dropout)          (None, 5, 50)             0
_________________________________________________________________
lstm_3 (LSTM)                (None, 200)               200800
_________________________________________________________________
batch_normalization_4 (Batch (None, 200)               800
_________________________________________________________________
dropout_4 (Dropout)          (None, 200)               0
_________________________________________________________________
dense_2 (Dense)              (None, 25)                5025
_________________________________________________________________
batch_normalization_5 (Batch (None, 25)                100
_________________________________________________________________
dropout_5 (Dropout)          (None, 25)                0
_________________________________________________________________
dense_3 (Dense)              (None, 1)                 26
=================================================================
Total params: 704,201
Trainable params: 702,851
Non-trainable params: 1,350
_________________________________________________________________
Time taken: 0:00:01.914407


But , I am seeing RMSE value for test data is getting very high. This is my train and validation loss graph.

What am I doing wrong here ? Is my model suffering from overfitting problem ?

Please help .

## 1 Answer

A high RMSE on the test set with a small RMSE on the train set is a sign of overfitting. Your plot looks weird, as there's no sign of overfitting on the validation set (I suppose that the label test means validation following your text). This might be caused by:

• your validation data doesn't represent your test data, e.g. they come from different distributions, the train/validation/test splits have not been carried out correctly, etc.
• you are overfitting the validation set, that is you have tweaked the parameters so many times based on what you have seen on the validation set that the RMSE on that specific set is almost perfect, but it doesn't generalize well on other sets

I suggest to use the usual techniques to prevent overfitting: start with a simpler architecture (fewer layers and nodes) and build from there, use cross-validation to adjust the dropout probability, perhaps include other forms of regularization, and so on.