2
$\begingroup$

While I am training, it seems like my loss is going down, but my accuracy remains constant throughout training. It always seems to go towards 0.0023 no matter how I tweak my network, input data length, etc.

My input data is a sequence of sample levels before a current sample level. So basically predict the current sample level based on the previous sample levels.

Network is Sequential:

Dense(seq_length)
LSTM(seq_length*10)
Dense(1)

Validation set is 0.33 of the total input samples.

Using Tensorflow Keras on Windows.

CSV of loss, accuracy, val loss, etc: https://pastebin.com/GPsmeUmg

$\endgroup$

2 Answers 2

2
$\begingroup$

Everything is fine, loss is going down, your model is learning. The problem is that you are using accuracy on a regression problem.

Accuracy should only be used for classification. It is a very common mistake to make when starting out. If you want another metric then you could use mean_absolute_error which, as the name suggest, tells you how large error your model makes on average.

The reason accuracy does not work is because it only works with exact matches where you have y_true == y_pred. In regression you are working with continuous values which makes this quite rare and will not have much to do with the performance of your model.

The reason you get any accuracy at all is likely because Keras does y_true == round(y_pred), rounding the model prediction. Otherwise accuracy would almost always be zero since the model will never get the same decimals. This question has a good answer by Esmailian that goes a bit more into details on this.

$\endgroup$
2
  • $\begingroup$ Thanks! I was already using that, but also accuracy and also validating the data. So you're saying accuracy is not important for what I am trying to do? The training seems to kind of do what I want, just wondering why the accuracy stays pretty much pinned at that specific value after the first epoch :) $\endgroup$ May 4, 2019 at 21:48
  • $\begingroup$ Yes, accuracy only makes sense for exact matches which is quite rare in regression. Will try to explain why in my answer. But if accuracy was the only issue, then all is good! :) $\endgroup$ May 4, 2019 at 21:52
1
$\begingroup$

I am assuming you are trying to do classification task using LSTM and your validation accuracy is not increasing.

  1. Your model is learning your training data only & making adjustment in the model weights according to it. This is called overfitting in machine learning.
  2. So the question is why overfitting happens one of the reason is the limited amount of data is exposed to the model.

I think this is the reason you model is not learning i.e. the limited amount of data.

$\endgroup$
1
  • $\begingroup$ So I am trying to train a RNN to reproduce the sound it is given. In my trials i am trying with a simple sine wave tone, which I doubt having more data would be the cause here. The Validation seems to always go to the specific 0.0023, even if it becomes higher or lower for a few batches. When training for a few hundred epochs, it kind of learns to reproduce it... so it does work. It's just the accuracy metric seems weird and I am trying to understand why :( $\endgroup$ May 4, 2019 at 12:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.