# Seems like the Keras .fit and .evaluate methods give different training accuracy (yet the same validation accuracy). Same thing on loss.?

The code below is a translation of Nielsen's first mnist code to Keras. Surprisingly, the last accuracy value of the .fit method and the accuracy value for the .evaluate method are different for the training data. As I expected, for the validation data the accuracies are the same. The same behavior is seen with losses. I would appreciate finding out what is going on.

import numpy as np
from tensorflow import keras
from tensorflow.keras import layers

num_classes = 10
input_shape = (28, 28, 1)

# the data, split between train and test sets
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
print(x_train.shape,y_train.shape,x_test.shape,y_test.shape)
# Scale images to the [0, 1] range
x_train = x_train.astype("float32") / 255
x_test = x_test.astype("float32") / 255
# Make sure images have shape (28, 28, 1)
x_train = np.expand_dims(x_train, -1)  #adds another dimension at the end for color
x_test = np.expand_dims(x_test, -1)

# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

initializer = keras.initializers.RandomNormal(mean=0., stddev=1.)

model = keras.Sequential(
[
keras.Input(shape=input_shape),
layers.Flatten(),
layers.Dense(30, activation="sigmoid", use_bias=True,
kernel_initializer=initializer,
bias_initializer=initializer),
layers.Dense(num_classes, activation="sigmoid",
kernel_initializer=initializer,
bias_initializer=initializer),
]
)
batch_size = 10
epochs = 2
opt = keras.optimizers.SGD(learning_rate=3)
model.compile(loss="MeanSquaredError", optimizer=opt, metrics=["accuracy"])
ann=model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=1/6)

# Evaluate the trained model
print("\n")
score = model.evaluate(x_train[:50000], y_train[:50000], verbose=0)
print("evaluate train accuracy:", score[1])
score = model.evaluate(x_train[-10000:], y_train[-10000:], verbose=0)
print("evaluate validation accuracy:", score[1])
print("history last train accuracy:",ann.history['accuracy'][-1])
print("history last validation accuracy:",ann.history['val_accuracy'][-1])

• Can you elaborate on what you refer to with “last accuracy”? – hH1sG0n3 Apr 11 at 8:54

## 1 Answer

training's loss/accuracy are not calculated for the whole dataset every time.
It is calculated for the current batch and averaged successively.
test's loss/accuracy is calculated at the end of an epoch for the whole test data.

Check these references
Keras FAQ
SO Answer

You may put the validation data same as the train data to check it.

ann=model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_data=(x_train,y_train))

# Evaluate the trained model
print("\n")
score = model.evaluate(x_train, y_train, verbose=0)


Or, you may write a custom callback to print appropriate logs as suggested in the SO answer.

• Many thanks 10xAI. I had not understood your answer. I incorrectly thought that Keras in model.fit first splits off the validation data and then takes the specified batch_size from the remaining training data. It first takes the specified batch-size and then splits off the validation data. In recall mode (model.evaluate) there is no splitting. This explains everything. My question has been satisfactorily answered and the thread should be closed with credit to 10xAI. – ProfTeachingNumericalMethods Apr 22 at 16:03
• That's great. Please accept this answer and will suggest you to close/delete the other question. – 10xAI Apr 22 at 16:12
• Unfortunately, I jumped the gun. Keras does first split off the validation data and then takes the specified batch_size from the remaining training data. – ProfTeachingNumericalMethods Apr 22 at 18:47
• But let's close this thread. I will simplify the code more to make the issue crystal clear – ProfTeachingNumericalMethods Apr 22 at 21:57