Regression in Keras

Question

I was trying to implement a regression model in Keras, but am unable to figure out how to calculate the score of my model, i.e., how well it performed on my dataset.

import numpy as np
import pandas as pd
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.cross_validation import cross_val_score, KFold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline


## Load the dataset
dataframe = pd.read_csv("housing.csv", delim_whitespace=True,header=None)
dataset = dataframe.values
X_train = dataset[:400,0:13]
Y_train = dataset[:400,13]
X_test = dataset[401:,0:13]
Y_test = dataset[401:,13]

##define base model
def base_model():
     model = Sequential()
     model.add(Dense(14, input_dim=13, init='normal', activation='relu'))
     model.add(Dense(7, init='normal', activation='relu'))
     model.add(Dense(1, init='normal'))
     model.compile(loss='mean_squared_error', optimizer = 'adam')
     return model

seed = 7
np.random.seed(seed)

scale = StandardScaler()
X_train = scale.fit_transform(X_train)
X_test = scale.fit_transform(X_test)

clf = KerasRegressor(build_fn=base_model, nb_epoch=100, batch_size=5,verbose=0)

clf.fit(X_test,Y_test)
res = clf.predict(X_test)

## line below throws an error
clf.score(Y_test,res)

Please tell me how can I get the score for my model and what mistake am I doing in the above code.

Answer 1

The syntax is not exact, you should pass the features X_test and the true labels Y_test to clt.score (the method performs the prediction on itself, no need to do it explicitly).

score = clf.score(X_test, Y_test)

You can also use other metrics available in the metrics module of sklearn. For example,

from sklearn.metrics import mean_squared_error
score = mean_squared_error(Y_test, clf.predict(X_test))

from sklearn.metrics import mean_absolute_error
score = mean_absolute_error(Y_test, clf.predict(X_test))

Just some other remarks on your code that are not directly related to the question:

• you should not call clf.fit on the test data, you should instead fit on the training data and use the test set to compute the score to check the generalization of your model

• you should fit StandardScaler only on the training data and use X_test = scale.transform(X_test) to apply the same transformation on the test set