I setup my neural net to use mean square error as shown below. To my understanding (and from reading the documentation) this means that if the correct result of a row is 0.7 and the net predicts 0.8 the contribution to the loss by this entry is (0.8 - 0.7) squared = 0.01
from keras.models import Sequential from keras.layers import Dense #...build up neural network layers here... net.compile(optimizer = 'adam', loss = 'mean_squared_error') net.fit(training_data, training_results, batch_size = 4, epochs = 100)
I get the following output.
Epoch 100/100 1190/1190 [==============================] - 0s 133us/step - loss: 0.0082
Wow the the loss is tiny, my little neural network is doing so well! However if I validate the result on my original training data
prediction = net.predict(training_data) prediction_delta = (prediction - training_results)
Although some of the values in the prediction_delta are small overall the loss is way higher than 0.0082 with single values as high as 0.44. Note that this is for the same training data used to fit the net and is not the test data (which also shows similar results) so I would expect to get the value 0.082 back. How does Keras calculate this loss number?