I am training a generic LSTM based autoencoder to get the sentence embeddings, the bleu score is the accuracy metric. The model is coded to output the same number of tokens as the length of labels, hence the losses are calculated using cross-entropy loss between the output of token and the corresponding label token and added to a total loss to be backpropagated The embeddings size is 1000 throughout.
Here are the logs:-
Training Epoch: 1/20, Training Loss: 78.32446559076034, Training Accuracy: 0.23442341868755373
Validation Epoch: 1/20, Validation Loss: 75.11487170562003, Validation Accuracy: 0.28851715943634565
Training Epoch: 2/20, Training Loss: 60.702940691499734, Training Accuracy: 0.3043263558919579
Validation Epoch: 2/20, Validation Loss: 68.58432596068359, Validation Accuracy: 0.337459858582381
Training Epoch: 3/20, Training Loss: 51.62519727157313, Training Accuracy: 0.35618672202599283
Validation Epoch: 3/20, Validation Loss: 64.17064862141332, Validation Accuracy: 0.37158793060235135
Training Epoch: 4/20, Training Loss: 44.40417488866389, Training Accuracy: 0.4094415453046547
Validation Epoch: 4/20, Validation Loss: 61.230048799977716, Validation Accuracy: 0.3955376494828317
Training Epoch: 5/20, Training Loss: 38.78325418571326, Training Accuracy: 0.46050421873328257
Validation Epoch: 5/20, Validation Loss: 59.78918521062842, Validation Accuracy: 0.4063787247291398
Training Epoch: 6/20, Training Loss: 33.65953556655257, Training Accuracy: 0.5193937894102788
Validation Epoch: 6/20, Validation Loss: 58.64455007580877, Validation Accuracy: 0.41980867690343776
Training Epoch: 7/20, Training Loss: 29.35849161540994, Training Accuracy: 0.5831378755700898
Validation Epoch: 7/20, Validation Loss: 58.26881152131025, Validation Accuracy: 0.4261582422867802
Training Epoch: 8/20, Training Loss: 25.244888168760856, Training Accuracy: 0.6488748581642462
Validation Epoch: 8/20, Validation Loss: 57.62903963564669, Validation Accuracy: 0.43286079887479756
Training Epoch: 9/20, Training Loss: 22.05663261861035, Training Accuracy: 0.7039174093261202
Validation Epoch: 9/20, Validation Loss: 58.09752491926684, Validation Accuracy: 0.4399501875046306
Training Epoch: 10/20, Training Loss: 19.248559526880282, Training Accuracy: 0.7486352249548112
Validation Epoch: 10/20, Validation Loss: 58.613073462421454, Validation Accuracy: 0.4470900014647744
Training Epoch: 11/20, Training Loss: 16.95602631587501, Training Accuracy: 0.7857343322245365
Validation Epoch: 11/20, Validation Loss: 58.38435334806304, Validation Accuracy: 0.44778823347334884
Training Epoch: 12/20, Training Loss: 14.74661236426599, Training Accuracy: 0.8136944976817879
Validation Epoch: 12/20, Validation Loss: 59.63633590068632, Validation Accuracy: 0.45206057264928495
Training Epoch: 13/20, Training Loss: 13.507415059699248, Training Accuracy: 0.8299945959036563
Validation Epoch: 13/20, Validation Loss: 60.149887264208886, Validation Accuracy: 0.4512303133278385
Training Epoch: 14/20, Training Loss: 12.026118357521792, Training Accuracy: 0.8491757446561087
Validation Epoch: 14/20, Validation Loss: 59.89944394497038, Validation Accuracy: 0.45497359431776685
Training Epoch: 15/20, Training Loss: 10.785567499923806, Training Accuracy: 0.8628473173326144
Validation Epoch: 15/20, Validation Loss: 61.482036528946125, Validation Accuracy: 0.45541000266481596
Training Epoch: 16/20, Training Loss: 9.373574649788727, Training Accuracy: 0.8767987081840235
Validation Epoch: 16/20, Validation Loss: 62.18386231796834, Validation Accuracy: 0.4580630794998584
Training Epoch: 17/20, Training Loss: 8.5658748998932, Training Accuracy: 0.8878869616990712
Validation Epoch: 17/20, Validation Loss: 63.56435154233743, Validation Accuracy: 0.4606744393166781
Training Epoch: 18/20, Training Loss: 7.807730126944895, Training Accuracy: 0.8960175152587504
Validation Epoch: 18/20, Validation Loss: 63.88373188037895, Validation Accuracy: 0.4606897915210869
Training Epoch: 19/20, Training Loss: 6.829077819740428, Training Accuracy: 0.9038927070366026
Validation Epoch: 19/20, Validation Loss: 65.59262917371629, Validation Accuracy: 0.4639800374912485
Training Epoch: 20/20, Training Loss: 6.152266260986982, Training Accuracy: 0.9090036335609419
Validation Epoch: 20/20, Validation Loss: 66.84154795008956, Validation Accuracy: 0.4672414105594907
Here is are the accuracy and loss vs epoch graphs :
I want to know why it is that the validation loss and accuracy is increasing.