0
$\begingroup$

There are several possibilites to evaluate a model:

hist = model.fit(x_train, y_train,
          (...)
          validation_data=(x_test, y_test))

or to use .evaluate() resp. .predict(). As far as I know I would use X_test and y_test for validation and prediction.

My question: How are they different? When I .fit() and .evaluate() afterwards, which information do I gain by .fit(validation_data(...))?

$\endgroup$
1
$\begingroup$

.fit --> You can give Train data and Validation data as well so Train data is used for Training and Validation data is used for validating the Trained model at the end of each epoch.

.predict --> It takes the only X and gives the output of the model.

.evaluate --> It will take X, Y and will evaluate similar to an above evaluation in .fit but after training total model we will use this.

You can write .evaluate = loss_function(.predict, y_true), score(.predict, y_true)

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks a lot! So in a practical manner I would .fit(..validation_data()) e.g. with a 80/20 split. And then I .evaluate() afterwards on all data? $\endgroup$ – Ben Oct 17 '19 at 11:25
  • 1
    $\begingroup$ @Ben yes that you can do, or you can create another test data to evaluate your model i.e. splitting data into train, val, test. $\endgroup$ – Uday Oct 17 '19 at 11:28
  • $\begingroup$ Thank you! When I use fit(..validation_data()) will the split be arbitrary or will the last 20 % be separated (I have time series data..). Furthermore, I used GridSearchCV in advance to find the best hyperparameters, does this change anything? E.g. do I have to create a training set, where I find the hyperparameters through GridSearchCV, then I use a separated val set to make use of fit(..validation_data())? $\endgroup$ – Ben Oct 17 '19 at 11:36
  • 1
    $\begingroup$ But for timeseries data, if you do random splitting, that is wrong. You have to use time series split only while doing gridsearch. Please check that. $\endgroup$ – Uday Oct 17 '19 at 12:47
  • $\begingroup$ Thanks, I'm already doing so. $\endgroup$ – Ben Oct 17 '19 at 12:50
1
$\begingroup$

As a practice data is divided into 2 parts first, train and test. Now test is kept completely separate and from train further 80 20 split is done as train and validation data.

Now when you fit model, for example say neural network model. For each epoch model will be trained using only and only train data. Test and validation are untouched. But at each epoch you want to know if the model trained is a better than previous or not (based on loss or any other metric you define), for that we evaluate performance on validation data (for this purpose only we give validation data in .fit)

Now as we are using validation data for checking best model, validation data is kind of used in training the model and hence to report final accuracy number of your model you use .predict on test data. Here also you can use .evaluate which returns predicted probability as well as loss contrary to .predict which just returns predicted probability.

Sometimes checking loss is also helpful as it gives sort of idea on how confident the model is in making predictions.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.