7
$\begingroup$

I was trying to implement a regression model in Keras, but am unable to figure out how to calculate the score of my model, i.e., how well it performed on my dataset.

import numpy as np
import pandas as pd
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.cross_validation import cross_val_score, KFold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline


## Load the dataset
dataframe = pd.read_csv("housing.csv", delim_whitespace=True,header=None)
dataset = dataframe.values
X_train = dataset[:400,0:13]
Y_train = dataset[:400,13]
X_test = dataset[401:,0:13]
Y_test = dataset[401:,13]

##define base model
def base_model():
     model = Sequential()
     model.add(Dense(14, input_dim=13, init='normal', activation='relu'))
     model.add(Dense(7, init='normal', activation='relu'))
     model.add(Dense(1, init='normal'))
     model.compile(loss='mean_squared_error', optimizer = 'adam')
     return model

seed = 7
np.random.seed(seed)

scale = StandardScaler()
X_train = scale.fit_transform(X_train)
X_test = scale.fit_transform(X_test)

clf = KerasRegressor(build_fn=base_model, nb_epoch=100, batch_size=5,verbose=0)

clf.fit(X_test,Y_test)
res = clf.predict(X_test)

## line below throws an error
clf.score(Y_test,res)

Please tell me how can I get the score for my model and what mistake am I doing in the above code.

$\endgroup$
1
  • $\begingroup$ where to download your housing.csv file to be implemted in keras REgressor???? $\endgroup$
    – deepak
    Jul 11, 2018 at 10:07

1 Answer 1

7
$\begingroup$

The syntax is not exact, you should pass the features X_test and the true labels Y_test to clt.score (the method performs the prediction on itself, no need to do it explicitly).

score = clf.score(X_test, Y_test)

You can also use other metrics available in the metrics module of sklearn. For example,

from sklearn.metrics import mean_squared_error
score = mean_squared_error(Y_test, clf.predict(X_test))

from sklearn.metrics import mean_absolute_error
score = mean_absolute_error(Y_test, clf.predict(X_test))

Just some other remarks on your code that are not directly related to the question:

  • you should not call clf.fit on the test data, you should instead fit on the training data and use the test set to compute the score to check the generalization of your model

  • you should fit StandardScaler only on the training data and use X_test = scale.transform(X_test) to apply the same transformation on the test set

$\endgroup$
2
  • $\begingroup$ @tlorieul When I changed my code to this : clf.ebaluate(X_test,Y_test). This code is throwing an error ''KerasRegressor' object has no attribute 'evaluate'' $\endgroup$
    – enterML
    Aug 10, 2016 at 12:01
  • $\begingroup$ @Nain Sorry, my bad, I did not read properly your code, I edited my answer, using Y_test instead of res is the answer. And as said by @NeilSlater, you shouldn't fit on the test data in any cases, you should fit on the train data, compute the score on the train data to check if the model was fitted correctly and compute the score on the test data and compare it to the training score to check if it has overfitted or not. $\endgroup$
    – tlorieul
    Aug 10, 2016 at 13:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.