I'm trying to train a text classifier using pytorch and the model currently uses pretrained embeddings, a bi-lstm followed by a linear layer and dropout. When I start training the loss is at 60-70, and gradually decreases to under 20 by epoch 30. However it just fluctuates in mid teens after that, by epoch 80 it's just hovering around 13-15 range. Does anyone have an idea what I might be doing wrong here?

I have a 20% val batch being tested on each epoch and the accuracy just stays at around 35% during the whole training session. Doesn't seem to change at all.

Would appreciate any pointers, thanks!

  • $\begingroup$ As mentioned the loss fluctuates between 15 and 16, maybe it is stuck at a local minimum? Since the validation accuracy is low, the model is stuck to a local minimum. It is still hard to tell without reviewing the architecture of your model. $\endgroup$ – Shubham Panchal Nov 3 '19 at 13:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.