Why does MAE differ after prediction (Neural Network)?

Question

I'm having trouble understanding what's happening in the following code. I already have defined x_train, y_train, x_val, y_val and x_test which define my training, validation and test sets. I'm using keras library.

model = Sequential()
model.add(Dense(8,input_dim=x_train.shape[1], activation='relu'))
model.add(Dense(1, activation='linear'))
model.summary()

rms = optimizers.RMSprop(lr=0.0003)

model.compile(optimizer=rms,
              loss='mean_squared_error',
              metrics=['mae'])
history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_val, y_val), batch_size = 32)


pred_train = model.predict(x_train)
pred_val = model.predict(x_val)
pred_test = model.predict(x_test) 


MAE_train = np.mean(np.absolute(y_train - pred_train ))
print('Mean absolute error for  training is MAE = '+str(MAE_train))
MAPE_train = 100 * np.mean(np.absolute(y_train - pred_train )/ y_train)
print('MAPE for training is MAPE = '+str(MAPE_train))
print(pred_val)
MAE_val= np.mean(np.absolute(y_val - pred_val ))
print('MAE  val is MAE = '+str(MAE_val))
MAPE_val = 100 * np.mean(np.absolute(y_val - pred_val )/ y_val)
print('MAPE for val is MAPE = '+str(MAPE_val))

here is the result of an execution :

What I don't understand is that the val_mean_absolute_error that is shown at the end of the training differs from the one that I compute. At first I tried to save my best model using Callbacks and then predict with it but the values also differ in this case.

What am i missing ?

Answer 1

Your MAE_val is:

MAE_val= np.mean(np.absolute(y_val - pred_val ))

On the other side, you fit your model on:

history = model.fit(x_train, y_train, ...

So you are calculating them on different objects. Training data on one side, and validation set for a final evaluation.