1
$\begingroup$

I don't know why I am getting such good results. 

Epoch 3/10 2937/2937 [==============================] - 12s 4ms/step -
loss: 0.2836 - acc: 0.4679 - val_loss: 0.1937 - val_acc: 0.1980

Epoch 4/10 2937/2937 [==============================] - 12s 4ms/step -
loss: 0.1355 - acc: 0.4679 - val_loss: 0.0866 - val_acc: 0.1980 

>Epoch
5/10 2937/2937 [==============================] - 13s 4ms/step - loss:
0.0580 - acc: 0.4679 - val_loss: 0.0342 - val_acc: 0.1980

Epoch 6/10 2937/2937 [==============================] - 13s 4ms/step -
loss: 0.0223 - acc: 0.4679 - val_loss: 0.0120 - val_acc: 0.1980

Epoch 7/10 2937/2937 [==============================] - 14s 5ms/step -
loss: 0.0082 - acc: 0.4679 - val_loss: 0.0040 - val_acc: 0.1980

My training and label sets are float number arrays in range [-0.05, 0.05] and I am using Keras.sequential.model.lstm. Why might this be happening? Previously, I had the opposite problem here: loss/val_loss are decreasing but accuracies are the same in LSTM!, but I couldn't understand the problem.

EDIT: I changed my code from:

model.compile(optimizer = 'adam', loss = 'mean_square_error', metrics=['accuracy'])

to:

model.compile(optimizer = 'adam', loss = 'mean_absolute_error', metrics=['accuracy'])

But the result is same.

I then changed the above line of the code to:

model.compile(optimizer = 'adam', loss = 'mean_squared_error', metrics=['mean_squared_error'])

But it didn't work and the result is as follow:

Train on 2937 samples, validate on 735 samples Epoch 1/10 2937/2937
[==============================] - 90s 31ms/step - loss: 1.6645 -
mean_squared_error: 0.0019 - val_loss: 0.7620 -
val_mean_squared_error: 0.0010

Epoch 2/10 2937/2937 [==============================] - 13s 4ms/step -
loss: 0.5503 - mean_squared_error: 0.0019 - val_loss: 0.3890 -
val_mean_squared_error: 0.0010

Epoch 3/10 2937/2937 [==============================] - 13s 4ms/step -
loss: 0.2837 - mean_squared_error: 0.0019 - val_loss: 0.1938 -
val_mean_squared_error: 0.0010

Epoch 4/10 2937/2937 [==============================] - 13s 4ms/step -
loss: 0.1355 - mean_squared_error: 0.0019 - val_loss: 0.0866 -
val_mean_squared_error: 0.0010

Epoch 5/10 2937/2937 [==============================] - 13s 4ms/step -
loss: 0.0580 - mean_squared_error: 0.0019 - val_loss: 0.0342 -
val_mean_squared_error: 0.0010

Epoch 6/10 2937/2937 [==============================] - 13s 4ms/step -
loss: 0.0223 - mean_squared_error: 0.0019 - val_loss: 0.0120 -
val_mean_squared_error: 0.0010

Epoch 7/10 2937/2937 [==============================] - 13s 5ms/step -
loss: 0.0082 - mean_squared_error: 0.0019 - val_loss: 0.0040 -
val_mean_squared_error: 0.0010

Epoch 8/10 2937/2937 [==============================] - 14s 5ms/step -
loss: 0.0035 - mean_squared_error: 0.0019 - val_loss: 0.0017 -
val_mean_squared_error: 0.0010

Epoch 9/10 2937/2937 [==============================] - 13s 5ms/step -
loss: 0.0022 - mean_squared_error: 0.0019 - val_loss: 0.0011 -
val_mean_squared_error: 0.0010

Epoch 10/10 2937/2937 [==============================] - 13s 5ms/step
- loss: 0.0019 - mean_squared_error: 0.0019 - val_loss: 0.0010 - val_mean_squared_error: 0.0010
Improve this question
$\endgroup$
6
  • 5
    $\begingroup$ Possible duplicate of loss/val_loss are decreasing but accuracies are the same in LSTM! $\endgroup$
    – Simon Larsson
    Apr 13 '19 at 10:21
  • $\begingroup$ You have already asked this and got it answered. You should stop using accuracy when your labels are floats, it does not work. If you want a metric your can try mean_absolute_error. It will tell you how big your error is on average, so smaller is better. $\endgroup$
    – Simon Larsson
    Apr 13 '19 at 10:26
  • $\begingroup$ @SimonLarsson: This is a opposite question as you see! $\endgroup$
    – WDR
    Apr 13 '19 at 14:40
  • 1
    $\begingroup$ No, it is the same. You are still using accuracy on regression (floats) which does not work. Your problem is still that train and val accuracy does not change and that is because you should not use accuracy. $\endgroup$
    – Simon Larsson
    Apr 13 '19 at 14:49
  • $\begingroup$ @SimonLarsson: I use model.compile(optimizer = 'adam', loss = 'mean_absolute_error', metrics=['accuracy']) But the result is same! $\endgroup$
    – WDR
    Apr 13 '19 at 14:52
1
$\begingroup$

In your code:

model.compile(optimizer = 'adam', loss = 'mean_absolute_error', metrics=['accuracy'])

You are using accuracy as a metric, metrics=['accuracy']. This does not work when you are doing regression (which you are doing). Regression is when you have continuous (floats) labels.

So instead your code should look like:

model.compile(optimizer = 'adam', loss = 'mean_squared_error', metrics=['mean_absolute_error'])

Then when you train you should look for both loss and your metric to be decreasing during training. If they are, then your network is behaving normally.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Your solution didn't work. I edited my question. $\endgroup$
    – WDR
    Apr 13 '19 at 17:47
  • 2
    $\begingroup$ It fixed an error you had which prevented you from debugging further or getting help from others. As you can see others have pointed out the same problem. So I would say that it did work, sorry that it did not get you all the way. $\endgroup$
    – Simon Larsson
    Apr 15 '19 at 17:24
0
$\begingroup$

Accuracy is a measure of classification performance.

Mean absolute error and mean square error are measures of regression performance.

Given you are predicting a range of values [-0.05, 0.05], you are performing regression. Accuracy is a meaningless measure for regression and should be ignored.

Improve this answer
$\endgroup$
1
  • $\begingroup$ You can see in EDIT2 part of the question I changed the metrics to model.compile(optimizer = 'adam', loss = 'mean_squared_error', metrics=['mean_squared_error']) but the result was the same. $\endgroup$
    – WDR
    Apr 14 '19 at 4:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.