I have written a LSTM network. It seems all the things are OK but when I train the network, I get the same loss amount about 4.9e-4 for every iterations! What is the problem? Why my network can't decrease the loss amount?
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
IF EVERYTHING ELSE IS OKAY then ideally, your loss should decrease every epoch which means your model is learning and updating weights accordingly.
Steady loss might mean that your model has over-fitted data. Your model has memorized data and there is no room for improvement. Maybe
1) You have less data with very complex LSTM. Try having very simple LSTM.
2) Apply regularization layer in between.
