0
$\begingroup$

This is my training log for ten epoch for a sentiment analysis model:

Train on 5487 samples, validate on 610 samples
Epoch 1/10
5487/5487 [==============================] - 23s 4ms/sample - loss: 1.4769 - accuracy: 0.5216 - val_loss: 2.4135 - val_accuracy: 0.6164
Epoch 2/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 7.5815 - accuracy: 0.4593 - val_loss: 9.6993 - val_accuracy: 0.3000
Epoch 3/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.8212 - accuracy: 0.3807 - val_loss: 9.4066 - val_accuracy: 0.3164
Epoch 4/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.6174 - accuracy: 0.3594 - val_loss: 9.4066 - val_accuracy: 0.3066
Epoch 5/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.5968 - accuracy: 0.3548 - val_loss: 9.4066 - val_accuracy: 0.3066
Epoch 6/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.5939 - accuracy: 0.3561 - val_loss: 9.4066 - val_accuracy: 0.3066
Epoch 7/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.5792 - accuracy: 0.3465 - val_loss: 9.4066 - val_accuracy: 0.3066
Epoch 8/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.6086 - accuracy: 0.3506 - val_loss: 9.4066 - val_accuracy: 0.3066
Epoch 9/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.6233 - accuracy: 0.3501 - val_loss: 9.4066 - val_accuracy: 0.3066
Epoch 10/10
5487/5487 [==============================] - 19s 3ms/sample - loss: 9.5821 - accuracy: 0.3548 - val_loss: 9.4066 - val_accuracy: 0.3066

And this is the model itself:

model = tf.keras.Sequential([
            tf.keras.layers.Embedding(input_dim=len(idx_to_word), output_dim=300),
            tf.keras.layers.Dropout(0.2,noise_shape=[None,50,1]),
            tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(512, use_bias=False)),
            tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(512, recurrent_dropout=0.2,
                                         dropout=0.2)),
            tf.keras.layers.Dense(len(label_to_idx))
        ])
model.compile(loss=tf.keras.losses.CategoricalCrossentropy(),
                  optimizer=tf.keras.optimizers.Adam(1e-3),
                  metrics=['accuracy'])
history = model.fit(  X, one_hot_labels,
    epochs=10,
    batch_size=64,
    validation_split=0.1,
    verbose=1,
    shuffle=True)

I wonder what is the reason for decreasing accuracy in my training proccess

$\endgroup$
3
  • $\begingroup$ what is your training script? $\endgroup$ – Bruno Lubascher May 1 '20 at 11:27
  • $\begingroup$ @BrunoLubascher added it $\endgroup$ – Marzi Heidari May 1 '20 at 11:30
  • $\begingroup$ everything seems fine. Did you try decreasing the optimizer's learning rate like suggested? $\endgroup$ – Bruno Lubascher May 1 '20 at 11:38
2
$\begingroup$

Decreasing accuracy as training progresses means that the learning rate for your model is too high. Your model weights are changing a lot due to the high learning rate and therefore moving away from the local minimum where your accuracy would be at its highest. Try decreasing the learning rate and see what happens.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.