4
$\begingroup$

I am trying to train a LSTM network, over a total of 200 epochs, with hidden layer size of 100 and 1 dense layer after the LSTM layer. I have used a batch size of 10 for the same. Basically, I am confused as to why the loss curve which I get (with MAE as loss criteria and Adam Optimiser) is looking very different from what a good model generally gives. I believe that the likely reason may be that the training is occurring over more number of epochs than should be ideal, and it is underfitting/overfitting, but I am not sure that how to recognise the same.

The loss curve for the model is Model Loss Curve

I would like to be sure of whether the model is overfitting or undercutting, and if I need to reduce the training epochs (say from 200 to 20?).Being new to this, is there any specific point to identify when to stop the training process (such as based on this loss curve). Any help in this regard is appreciated.

Improve this question
$\endgroup$
1
  • $\begingroup$ What is the size of training and validation sets on which the plotted errors are calculated? And what is the average of target values? e.g. error 0.1 is from |1000.1 - 1000| or |1.1 - 1.0|? $\endgroup$
    – Esmailian
    Commented Mar 27, 2019 at 14:56

1 Answer 1

Reset to default
1
$\begingroup$

Overfitting :

The model tries to memorize what it has learned. Hence, it could not classify unseen samples.

In case of overfitting, the validation accuracy stops increasing and the validation loss also does not decrease.

It means that the model can no more generalise itself to get a validation accuracy above a certain threshold.

Hence, you can stop the training, when the val_acc does not change for a specific number of epochs.

Underfitting :

Underfitting means that model is not able to classify any of the samples even after learning them.

The model should stop its training when the accuracy and loss seem to be constant or they only revolve around a certain value.

In your case :

The loss for the train as well as test seem to decreasing simultaneously. The test curve flattens a bit earlier. It could be treated if the learning rate is decreased.

Tip: If you are using Keras, try the EarlyStopping callback.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the answer. Just a small query, if I use # fit network keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=0, patience=0, verbose=0, mode='auto', baseline=None, restore_best_weights=False) # here I am fitting the LSTM model -- history = model.fit(train_X, train_y, epochs=100, batch_size= 10, validation_data=(test_X, test_y), verbose=2, shuffle=False), is this the right way to use early stopping? I mean, is the early stopping function in Keras called just before fitting the model? Thanks again. $\endgroup$
    – JChat
    Commented Mar 27, 2019 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.