keras validation mean squared error always similar to 1

Question

is there any reason why the validation mean squared error output from Keras is always very similar to 1? Thank you. All of my training results looks like:

155/155 [==============================] - 0s - loss: 6062.6136 - mean_absolute_error: 0.8344 - mean_squared_error: 1.0271 - val_loss: 0.8252 - val_mean_absolute_error: 0.8252 - val_mean_squared_error: 1.0164
Epoch 29/1000
155/155 [==============================] - 0s - loss: 5870.5280 - mean_absolute_error: 0.8324 - mean_squared_error: 1.0211 - val_loss: 0.8246 - val_mean_absolute_error: 0.8246 - val_mean_squared_error: 1.0130
Epoch 30/1000
155/155 [==============================] - 0s - loss: 5668.5083 - mean_absolute_error: 0.8311 - mean_squared_error: 1.0134 - val_loss: 0.8244 - val_mean_absolute_error: 0.8244 - val_mean_squared_error: 1.0106
Epoch 31/1000
155/155 [==============================] - 0s - loss: 5530.8119 - mean_absolute_error: 0.8288 - mean_squared_error: 1.0115 - val_loss: 0.8243 - val_mean_absolute_error: 0.8243 - val_mean_squared_error: 1.0089
Epoch 32/1000
155/155 [==============================] - 0s - loss: 5222.6773 - mean_absolute_error: 0.8283 - mean_squared_error: 1.0119 - val_loss: 0.8245 - val_mean_absolute_error: 0.8245 - val_mean_squared_error: 1.0071
Epoch 33/1000
155/155 [==============================] - 0s - loss: 5090.0273 - mean_absolute_error: 0.8273 - mean_squared_error: 1.0078 - val_loss: 0.8247 - val_mean_absolute_error: 0.8247 - val_mean_squared_error: 1.0060
Epoch 34/1000
155/155 [==============================] - 0s - loss: 4878.2420 - mean_absolute_error: 0.8272 - mean_squared_error: 1.0093 - val_loss: 0.8245 - val_mean_absolute_error: 0.8245 - val_mean_squared_error: 1.0046

note: I have standardized my input and output with sklearn standardization:

from sklearn import preprocessing
    X_scaler = preprocessing.StandardScaler().fit(X_list_total)
    X_list_total_standardized = X_scaler.transform(X_list_total)
    Y_scaler = preprocessing.StandardScaler().fit(Y_list_total)
    Y_list_total_standardized = Y_scaler.transform(Y_list_total)

Does it just mean that there is nothing to learn from the data at all?

Answer 1

It could be very close to 1 (or for that matter, changing very slowly) for many reasons:

• Maybe your learning rate is very small.
• Maybe the network has reached its learning capacity. Since we do not know anything about the network architecture, we can't rule this out.

This does not mean there is nothing to learn from the data. In the worst case, there is nothing to learn from data, with the given architecture, hyperparameters and the time you are willing to let the network learn.

Answer 2

What is your output? Is it a 0-1? If so, you shouldn't be using RMSE and should be using cross-entropy as your loss function.