4
$\begingroup$

I'm having trouble understanding what's happening in the following code. I already have defined x_train, y_train, x_val, y_val and x_test which define my training, validation and test sets. I'm using keras library.

model = Sequential()
model.add(Dense(8,input_dim=x_train.shape[1], activation='relu'))
model.add(Dense(1, activation='linear'))
model.summary()

rms = optimizers.RMSprop(lr=0.0003)

model.compile(optimizer=rms,
              loss='mean_squared_error',
              metrics=['mae'])
history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_val, y_val), batch_size = 32)


pred_train = model.predict(x_train)
pred_val = model.predict(x_val)
pred_test = model.predict(x_test) 


MAE_train = np.mean(np.absolute(y_train - pred_train ))
print('Mean absolute error for  training is MAE = '+str(MAE_train))
MAPE_train = 100 * np.mean(np.absolute(y_train - pred_train )/ y_train)
print('MAPE for training is MAPE = '+str(MAPE_train))
print(pred_val)
MAE_val= np.mean(np.absolute(y_val - pred_val ))
print('MAE  val is MAE = '+str(MAE_val))
MAPE_val = 100 * np.mean(np.absolute(y_val - pred_val )/ y_val)
print('MAPE for val is MAPE = '+str(MAPE_val))

here is the result of an execution :

enter image description here

What I don't understand is that the val_mean_absolute_error that is shown at the end of the training differs from the one that I compute. At first I tried to save my best model using Callbacks and then predict with it but the values also differ in this case.

What am i missing ?

$\endgroup$
  • $\begingroup$ What is the values that is puzzling you? 2.201449... ? $\endgroup$ – Leevo Jan 16 at 19:20
  • $\begingroup$ yes I think that it should be 1.0773 instead. The same could be said for MAE on training set $\endgroup$ – Shinra_SGr Jan 16 at 19:31
1
$\begingroup$

Your MAE_val is:

MAE_val= np.mean(np.absolute(y_val - pred_val ))

On the other side, you fit your model on:

history = model.fit(x_train, y_train, ...

So you are calculating them on different objects. Training data on one side, and validation set for a final evaluation.

| improve this answer | |
$\endgroup$
  • $\begingroup$ MAE_ val is computed on validation set yes, but it is also the case for val_mean_absolute_error in the print screen since i give validation as an argument of the fit function model.fit(x_train, y_train, epochs=1000, validation_data=(x_val, y_val), batch_size = 32) $\endgroup$ – Shinra_SGr Jan 16 at 19:28
  • $\begingroup$ that's why I think both values should be equal $\endgroup$ – Shinra_SGr Jan 16 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.