validation accuracy and loss increase

Question

I am training a generic LSTM based autoencoder to get the sentence embeddings, the bleu score is the accuracy metric. The model is coded to output the same number of tokens as the length of labels, hence the losses are calculated using cross-entropy loss between the output of token and the corresponding label token and added to a total loss to be backpropagated The embeddings size is 1000 throughout.

Here are the logs:-

Training Epoch: 1/20, Training Loss: 78.32446559076034, Training Accuracy: 0.23442341868755373
Validation Epoch: 1/20, Validation Loss: 75.11487170562003, Validation Accuracy: 0.28851715943634565

Training Epoch: 2/20, Training Loss: 60.702940691499734, Training Accuracy: 0.3043263558919579
Validation Epoch: 2/20, Validation Loss: 68.58432596068359, Validation Accuracy: 0.337459858582381

Training Epoch: 3/20, Training Loss: 51.62519727157313, Training Accuracy: 0.35618672202599283
Validation Epoch: 3/20, Validation Loss: 64.17064862141332, Validation Accuracy: 0.37158793060235135

Training Epoch: 4/20, Training Loss: 44.40417488866389, Training Accuracy: 0.4094415453046547
Validation Epoch: 4/20, Validation Loss: 61.230048799977716, Validation Accuracy: 0.3955376494828317

Training Epoch: 5/20, Training Loss: 38.78325418571326, Training Accuracy: 0.46050421873328257
Validation Epoch: 5/20, Validation Loss: 59.78918521062842, Validation Accuracy: 0.4063787247291398

Training Epoch: 6/20, Training Loss: 33.65953556655257, Training Accuracy: 0.5193937894102788
Validation Epoch: 6/20, Validation Loss: 58.64455007580877, Validation Accuracy: 0.41980867690343776

Training Epoch: 7/20, Training Loss: 29.35849161540994, Training Accuracy: 0.5831378755700898
Validation Epoch: 7/20, Validation Loss: 58.26881152131025, Validation Accuracy: 0.4261582422867802

Training Epoch: 8/20, Training Loss: 25.244888168760856, Training Accuracy: 0.6488748581642462
Validation Epoch: 8/20, Validation Loss: 57.62903963564669, Validation Accuracy: 0.43286079887479756

Training Epoch: 9/20, Training Loss: 22.05663261861035, Training Accuracy: 0.7039174093261202
Validation Epoch: 9/20, Validation Loss: 58.09752491926684, Validation Accuracy: 0.4399501875046306

Training Epoch: 10/20, Training Loss: 19.248559526880282, Training Accuracy: 0.7486352249548112
Validation Epoch: 10/20, Validation Loss: 58.613073462421454, Validation Accuracy: 0.4470900014647744

Training Epoch: 11/20, Training Loss: 16.95602631587501, Training Accuracy: 0.7857343322245365
Validation Epoch: 11/20, Validation Loss: 58.38435334806304, Validation Accuracy: 0.44778823347334884

Training Epoch: 12/20, Training Loss: 14.74661236426599, Training Accuracy: 0.8136944976817879
Validation Epoch: 12/20, Validation Loss: 59.63633590068632, Validation Accuracy: 0.45206057264928495

Training Epoch: 13/20, Training Loss: 13.507415059699248, Training Accuracy: 0.8299945959036563
Validation Epoch: 13/20, Validation Loss: 60.149887264208886, Validation Accuracy: 0.4512303133278385

Training Epoch: 14/20, Training Loss: 12.026118357521792, Training Accuracy: 0.8491757446561087
Validation Epoch: 14/20, Validation Loss: 59.89944394497038, Validation Accuracy: 0.45497359431776685

Training Epoch: 15/20, Training Loss: 10.785567499923806, Training Accuracy: 0.8628473173326144
Validation Epoch: 15/20, Validation Loss: 61.482036528946125, Validation Accuracy: 0.45541000266481596

Training Epoch: 16/20, Training Loss: 9.373574649788727, Training Accuracy: 0.8767987081840235
Validation Epoch: 16/20, Validation Loss: 62.18386231796834, Validation Accuracy: 0.4580630794998584

Training Epoch: 17/20, Training Loss: 8.5658748998932, Training Accuracy: 0.8878869616990712
Validation Epoch: 17/20, Validation Loss: 63.56435154233743, Validation Accuracy: 0.4606744393166781

Training Epoch: 18/20, Training Loss: 7.807730126944895, Training Accuracy: 0.8960175152587504
Validation Epoch: 18/20, Validation Loss: 63.88373188037895, Validation Accuracy: 0.4606897915210869

Training Epoch: 19/20, Training Loss: 6.829077819740428, Training Accuracy: 0.9038927070366026
Validation Epoch: 19/20, Validation Loss: 65.59262917371629, Validation Accuracy: 0.4639800374912485

Training Epoch: 20/20, Training Loss: 6.152266260986982, Training Accuracy: 0.9090036335609419
Validation Epoch: 20/20, Validation Loss: 66.84154795008956, Validation Accuracy: 0.4672414105594907

Here is are the accuracy and loss vs epoch graphs :

I want to know why it is that the validation loss and accuracy is increasing.

Answer 1

An increase in validation loss while training loss is decreasing is an indicator that your model overfits. Check out this article for an easy to read general explanation.

In the context of autoencoders this means your neural net almost reproduces the input image. Try to reduce overfit by applying regularization, e.g. add dropout, add input noise, use less layers or use less nodes per layer (not all at once but one by one).