I want to predict a time series with multiple variables. I am using Keras's LSTM class.
Here is my data set description :
I want to predict var1(t-1) and my X variables are var3(t-1) , var4(t-1) , var5(t-1) , var6(t-1) and var7(t-1).
And this is how my model configuration looks like :
{
"data": {
"columns": [
"var1(t-1)",
"var3(t-1)",
"var4(t-1)",
"var5(t-1)",
"var6(t-1)",
"var7(t-1)"
],
"sequence_length": 5,
"train_test_split": 0.80,
"normalise": false
},
"training": {
"epochs": 100,
"batch_size":10
},
"model": {
"loss": "mse",
"optimizer": "nadam",
"save_dir": "saved_models_multi",
"layers": [
{
"type": "lstm",
"neurons": 200,
"input_timesteps": 5,
"input_dim": 5,
"return_seq": true
},
{
"type":"batch_norm"
},
{
"type": "dropout",
"rate": 0.4
},
{
"type": "lstm",
"neurons": 200,
"return_seq": true
},
{
"type":"batch_norm"
},
{
"type": "dropout",
"rate": 0.4
},
{
"type": "dense",
"neurons": 50,
"activation": "sigmoid"
},
{
"type":"batch_norm"
},
{
"type": "dropout",
"rate": 0.2
},
{
"type": "lstm",
"neurons": 200,
"return_seq": false
},
{
"type":"batch_norm"
},
{
"type": "dropout",
"rate": 0.4
},
{
"type": "dense",
"neurons": 25,
"activation": "sigmoid"
},
{
"type":"batch_norm"
},
{
"type": "dropout",
"rate": 0.2
},
{
"type": "dense",
"neurons": 1,
"activation": "linear"
}
]
}
}
This is number of parameters in each layer and output shape :
Layer (type) Output Shape Param #
=================================================================
lstm_1 (LSTM) (None, 5, 200) 164800
_________________________________________________________________
batch_normalization_1 (Batch (None, 5, 200) 800
_________________________________________________________________
dropout_1 (Dropout) (None, 5, 200) 0
_________________________________________________________________
lstm_2 (LSTM) (None, 5, 200) 320800
_________________________________________________________________
batch_normalization_2 (Batch (None, 5, 200) 800
_________________________________________________________________
dropout_2 (Dropout) (None, 5, 200) 0
_________________________________________________________________
dense_1 (Dense) (None, 5, 50) 10050
_________________________________________________________________
batch_normalization_3 (Batch (None, 5, 50) 200
_________________________________________________________________
dropout_3 (Dropout) (None, 5, 50) 0
_________________________________________________________________
lstm_3 (LSTM) (None, 200) 200800
_________________________________________________________________
batch_normalization_4 (Batch (None, 200) 800
_________________________________________________________________
dropout_4 (Dropout) (None, 200) 0
_________________________________________________________________
dense_2 (Dense) (None, 25) 5025
_________________________________________________________________
batch_normalization_5 (Batch (None, 25) 100
_________________________________________________________________
dropout_5 (Dropout) (None, 25) 0
_________________________________________________________________
dense_3 (Dense) (None, 1) 26
=================================================================
Total params: 704,201
Trainable params: 702,851
Non-trainable params: 1,350
_________________________________________________________________
Time taken: 0:00:01.914407
But , I am seeing RMSE value for test data is getting very high. This is my train and validation loss graph.
What am I doing wrong here ? Is my model suffering from overfitting problem ?
Please help .
