I am training an LSTM for time series forecasting and it has produced an extremly high loss value during one epoch:
Epoch 00043: saving model to /...
904/904 - 2s - loss: 0.7537 - mean_absolute_error: 0.5772 - val_loss: 1.4430
- val_mean_absolute_error: 0.7124
Epoch 00044: saving model to /...
904/904 - 2s - loss: 240372339275.7649 - mean_absolute_error: 56354.0078
- val_loss: 4.6229 - val_mean_absolute_error: 1.5681
Epoch 00045: saving model to /...
904/904 - 2s - loss: 1.3348 - mean_absolute_error: 0.7894 - val_loss: 2.2875
- val_mean_absolute_error: 1.1510
My model:
model = keras.Sequential()
model.add(keras.layers.LSTM(360, activation='relu', input_shape=(N_STEPS, n_features)))
model.add(keras.layers.Dropout(0.1))
model.add(keras.layers.Dense(1, activation='relu'))
model.compile(optimizer='adam', loss='mse', metrics=['mae'])
What is the cause of this?
Theoretically, it shouldn't be able to have such a high loss unless it outputs very high values for that epoch. Which is strange since the model's output makes sense during other epochs.