# ANN applied to Boston Housing dataset returns negative value

This example is taken from the book Deep Learning With Python from Jason Brownlee. It applies a fully connected neural model with one hidden layer (13, 13, 1) using Keras library and the Tensorflow backend.

1 - Import the packages

import numpy
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline


boston = load_boston()
X = boston.data[:,0:13]
Y = boston.target


3 - Define base model

def baseline_model():
# create model
model = Sequential()
kernel_initializer='normal', activation='relu'))
# Compile model
return model


4 - Fix random seed for reproducibility

seed = 7
numpy.random.seed(seed)


5 - Fit & evaluate model

estimator = KerasRegressor(build_fn=baseline_model, epochs=10, \
batch_size=5, verbose=0)
kfold = KFold(n_splits=10, random_state=seed)
results = cross_val_score(estimator, X, Y, cv=kfold)


6 - Value of results parameter

Out[63]:
array([-13.89524042, -14.2215869 ,  -6.21156782, -42.65242339,
-26.58890147, -56.30860755, -28.6575911 , -89.67339525,
-27.7172946 , -22.67604859])


Which returns a mean negative value

print("Baseline: %.2f (%.2f) MSE" % (results.mean(), results.std()))


It returns about -30 and it should return about +30 according to the book. I've tried both Theano and Tensorflow with no success.

I've also tested this code both on Windows and Linux, having obtained the same bad result.

The problem seems to be in the cross-validation part, because if I don't run cross validation, I get more sensible results.

7 - Without cross-validation

model = Sequential()
activation='relu'))
model.fit(X, Y, epochs=150, batch_size=10)


Now if I evaluate the model I get a more sensible value

In [68]: model.evaluate(X, Y)
506/506 [==============================] - 0s 58us/step
Out[68]: 27.778296670423664


What can be happening here? Why is the cross-validation procedure returning negative values?

"This is to be expected. sklearn has changed their API to invert their cost functions. Nothing to be concerned about."

I got this answer from Dr. Jason Brownlee.