0
$\begingroup$

I want to predict a time series with multiple variables. I am using Keras's LSTM class.

Here is my data set description :

dataset

I want to predict var1(t-1) and my X variables are var3(t-1) , var4(t-1) , var5(t-1) , var6(t-1) and var7(t-1).

And this is how my model configuration looks like :

{
    "data": {

        "columns": [            
            "var1(t-1)",
            "var3(t-1)",
            "var4(t-1)",          
            "var5(t-1)",
            "var6(t-1)",
            "var7(t-1)" 
        ],
        "sequence_length": 5,
        "train_test_split": 0.80,
        "normalise": false
    },
    "training": {
        "epochs": 100,
        "batch_size":10 
    },
    "model": {
        "loss": "mse",
        "optimizer": "nadam",
        "save_dir": "saved_models_multi",
        "layers": [
            {
                "type": "lstm",
                "neurons": 200,
                "input_timesteps": 5,
                "input_dim": 5,
                "return_seq": true
            },
            {
              "type":"batch_norm"  

            },  
            {
                "type": "dropout",
                "rate": 0.4
            },

            {
                "type": "lstm",
                "neurons": 200,
                "return_seq": true
            },
            {
              "type":"batch_norm"  

            },

            {
                "type": "dropout",
                "rate": 0.4
            },
            {
                "type": "dense",
                "neurons": 50,
                "activation": "sigmoid"
            },

             {
              "type":"batch_norm"  

            },

            {
                "type": "dropout",
                "rate": 0.2
            },

            {
                "type": "lstm",
                "neurons": 200,
                "return_seq": false
            },
            {
              "type":"batch_norm"  

            },

            {
                "type": "dropout",
                "rate": 0.4
            },
            {
                "type": "dense",
                "neurons": 25,
                "activation": "sigmoid"
            },

             {
              "type":"batch_norm"  

            },

            {
                "type": "dropout",
                "rate": 0.2
            },

            {
                "type": "dense",
                "neurons": 1,
                "activation": "linear"
            }
        ]
    }
}

This is number of parameters in each layer and output shape :

Layer (type)                 Output Shape              Param #   
=================================================================
lstm_1 (LSTM)                (None, 5, 200)            164800    
_________________________________________________________________
batch_normalization_1 (Batch (None, 5, 200)            800       
_________________________________________________________________
dropout_1 (Dropout)          (None, 5, 200)            0         
_________________________________________________________________
lstm_2 (LSTM)                (None, 5, 200)            320800    
_________________________________________________________________
batch_normalization_2 (Batch (None, 5, 200)            800       
_________________________________________________________________
dropout_2 (Dropout)          (None, 5, 200)            0         
_________________________________________________________________
dense_1 (Dense)              (None, 5, 50)             10050     
_________________________________________________________________
batch_normalization_3 (Batch (None, 5, 50)             200       
_________________________________________________________________
dropout_3 (Dropout)          (None, 5, 50)             0         
_________________________________________________________________
lstm_3 (LSTM)                (None, 200)               200800    
_________________________________________________________________
batch_normalization_4 (Batch (None, 200)               800       
_________________________________________________________________
dropout_4 (Dropout)          (None, 200)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 25)                5025      
_________________________________________________________________
batch_normalization_5 (Batch (None, 25)                100       
_________________________________________________________________
dropout_5 (Dropout)          (None, 25)                0         
_________________________________________________________________
dense_3 (Dense)              (None, 1)                 26        
=================================================================
Total params: 704,201
Trainable params: 702,851
Non-trainable params: 1,350
_________________________________________________________________
Time taken: 0:00:01.914407

But , I am seeing RMSE value for test data is getting very high. This is my train and validation loss graph.

What am I doing wrong here ? Is my model suffering from overfitting problem ?

Please help .

enter image description here

$\endgroup$
0
$\begingroup$

A high RMSE on the test set with a small RMSE on the train set is a sign of overfitting. Your plot looks weird, as there's no sign of overfitting on the validation set (I suppose that the label test means validation following your text). This might be caused by:

  • your validation data doesn't represent your test data, e.g. they come from different distributions, the train/validation/test splits have not been carried out correctly, etc.
  • you are overfitting the validation set, that is you have tweaked the parameters so many times based on what you have seen on the validation set that the RMSE on that specific set is almost perfect, but it doesn't generalize well on other sets

I suggest to use the usual techniques to prevent overfitting: start with a simpler architecture (fewer layers and nodes) and build from there, use cross-validation to adjust the dropout probability, perhaps include other forms of regularization, and so on.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.