I'm trying to train a text classifier using pytorch and the model currently uses pretrained embeddings, a bi-lstm followed by a linear layer and dropout. When I start training the loss is at 60-70, and gradually decreases to under 20 by epoch 30. However it just fluctuates in mid teens after that, by epoch 80 it's just hovering around 13-15 range. Does anyone have an idea what I might be doing wrong here?
I have a 20% val batch being tested on each epoch and the accuracy just stays at around 35% during the whole training session. Doesn't seem to change at all.
Would appreciate any pointers, thanks!