0
$\begingroup$

So far I think I understood the differences between the training, test and validation set. Basically it is like in this image:

Overview

Training set: The data where the model is trained on

Validation set: Data the model has not been trained on and used to tune hyperparameters

Test set: In principle the same like the validation set.. just used at the final end after the model has been tailored.

The training set is usually set up via some cross validation. When fitting via model.fit(X_train, y_train,..) will Keras shuffle the data autonomously ?

Next, in Keras, you are able to provide the validation set inside the model.fit() method as validation_data=(x_test, y_test) but there is also the possibility to provide e.g. validation_split = 0.2 What is the difference?

And after that, the test/prediction set will be taken into account just as model.pred(X_pred, y_pred,..). Right?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

When fitting via model.fit(X_train, y_train,..), will Keras shuffle the data autonomously?

Yes. shuffle = True is default. So, it basically shuffles every time.

Next, in Keras, you are able to provide the validation set inside the model.fit() method as validation_data=(x_test, y_test) but there is also the possibility to provide e.g. validation_split = 0.2 What is the difference?

The difference is that you can manually provide a validation data set. It's not X_test, Y_test. Rather X_test and Y_test are used for model evaluation model.evaluate() or model.predict(). model.fit is used for the training dataset. When you say validation_set = 0.2, it takes 20% data from training dataset and provides you the validation accuracy and loss.

And after that, the test/prediction set will be taken into account just as model.fit(X_pred, y_pred,..). Right?

You need to use model.predict(X_pred, Y_pred)

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thanks a lot! Some more questions: E.g. on stackoverflow.com/questions/40263117/… the CV resp. the fitting is done within a for loop. How does Keras know in "model.fit()" which folds it has to use when it is not done inside a loop? Then: using "validation_set" instead of a separate data set is recommendable? And "model.evaluate()" is not necessary in this case, right? $\endgroup$
    – Ben
    Commented Oct 9, 2019 at 6:29
  • 1
    $\begingroup$ Keras basically knows it by 'for loop'. I think the code in link you mentioned is not correct. When you use K-fold cross-validation, K-1 folds will be used for training and some portion can be used for validation. The last fold should be used for testing. $\endgroup$
    – Bloodstone Programmer
    Commented Oct 11, 2019 at 8:06
  • $\begingroup$ Thanks, so this means I have to fit the model within a for-loop? $\endgroup$
    – Ben
    Commented Oct 11, 2019 at 8:46
  • $\begingroup$ You can use train-test- validation split (say 60% train, 20% validation and 20% test.) when you have large dataset. In that case you dont need to use for loop. For smaller dataset normally K-fold is used and in that case for loop can be applied. If you find this answer satisfactory, please don forget to mark right. Thank you $\endgroup$
    – Bloodstone Programmer
    Commented Oct 11, 2019 at 16:32
  • 1
    $\begingroup$ Yes. train_test_split(), you should not use a loop because you have one set of training and one set of testing data. But in Kfold(), say you use k = 5 that is means 5 fold cross-validation. In this case, you will get 5 equal set of data from where you want to use 4 set as tarining and the rest for tseting. Iterate this process for other set also. towardsdatascience.com/…. This post will help you to understand the cross-validation technique $\endgroup$
    – Bloodstone Programmer
    Commented Oct 15, 2019 at 22:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.