2
$\begingroup$

This is my training log for ten epoch for a sentiment analysis model:

Train on 5487 samples, validate on 610 samples
Epoch 1/10
5487/5487 [==============================] - 23s 4ms/sample - loss: 1.4769 - accuracy: 0.5216 - val_loss: 2.4135 - val_accuracy: 0.6164
Epoch 2/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 7.5815 - accuracy: 0.4593 - val_loss: 9.6993 - val_accuracy: 0.3000
Epoch 3/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.8212 - accuracy: 0.3807 - val_loss: 9.4066 - val_accuracy: 0.3164
Epoch 4/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.6174 - accuracy: 0.3594 - val_loss: 9.4066 - val_accuracy: 0.3066
Epoch 5/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.5968 - accuracy: 0.3548 - val_loss: 9.4066 - val_accuracy: 0.3066
Epoch 6/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.5939 - accuracy: 0.3561 - val_loss: 9.4066 - val_accuracy: 0.3066
Epoch 7/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.5792 - accuracy: 0.3465 - val_loss: 9.4066 - val_accuracy: 0.3066
Epoch 8/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.6086 - accuracy: 0.3506 - val_loss: 9.4066 - val_accuracy: 0.3066
Epoch 9/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.6233 - accuracy: 0.3501 - val_loss: 9.4066 - val_accuracy: 0.3066
Epoch 10/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.5821 - accuracy: 0.3548 - val_loss: 9.4066 - val_accuracy: 0.3066

And this is the model itself:

model = tf.keras.Sequential([
            tf.keras.layers.Embedding(input_dim=len(idx_to_word), output_dim=300),
            tf.keras.layers.Dropout(0.2,noise_shape=[None,50,1]),
            tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(512, use_bias=False)),
            tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(512, recurrent_dropout=0.2,
                                         dropout=0.2)),
            tf.keras.layers.Dense(len(label_to_idx))
        ])
model.compile(loss=tf.keras.losses.CategoricalCrossentropy(),
                  optimizer=tf.keras.optimizers.Adam(1e-3),
                  metrics=['accuracy'])
history = model.fit(  X, one_hot_labels,
    epochs=10,
    batch_size=64,
    validation_split=0.1,
    verbose=1,
    shuffle=True)

I wonder what is the reason for decreasing accuracy in my training proccess

$\endgroup$
3
  • $\begingroup$ what is your training script? $\endgroup$ May 1, 2020 at 11:27
  • $\begingroup$ @BrunoLubascher added it $\endgroup$ May 1, 2020 at 11:30
  • $\begingroup$ everything seems fine. Did you try decreasing the optimizer's learning rate like suggested? $\endgroup$ May 1, 2020 at 11:38

1 Answer 1

3
$\begingroup$

Decreasing accuracy as training progresses means that the learning rate for your model is too high. Your model weights are changing a lot due to the high learning rate and therefore moving away from the local minimum where your accuracy would be at its highest. Try decreasing the learning rate and see what happens.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.