# clarification on train, test and val and how to use/implement it

So far I think I understood the differences between the training, test and validation set. Basically it is like in this image:

Training set: The data where the model is trained on

Validation set: Data the model has not been trained on and used to tune hyperparameters

Test set: In principle the same like the validation set.. just used at the final end after the model has been tailored.

The training set is usually set up via some cross validation. When fitting via model.fit(X_train, y_train,..) will Keras shuffle the data autonomously ?

Next, in Keras, you are able to provide the validation set inside the model.fit() method as validation_data=(x_test, y_test) but there is also the possibility to provide e.g. validation_split = 0.2 What is the difference?

And after that, the test/prediction set will be taken into account just as model.pred(X_pred, y_pred,..). Right?

When fitting via model.fit(X_train, y_train,..), will Keras shuffle the data autonomously?

Yes. shuffle = True is default. So, it basically shuffles every time.

Next, in Keras, you are able to provide the validation set inside the model.fit() method as validation_data=(x_test, y_test) but there is also the possibility to provide e.g. validation_split = 0.2 What is the difference?

The difference is that you can manually provide a validation data set. It's not X_test, Y_test. Rather X_test and Y_test are used for model evaluation model.evaluate() or model.predict(). model.fit is used for the training dataset. When you say validation_set = 0.2, it takes 20% data from training dataset and provides you the validation accuracy and loss.

And after that, the test/prediction set will be taken into account just as model.fit(X_pred, y_pred,..). Right?

You need to use model.predict(X_pred, Y_pred)

• Thanks a lot! Some more questions: E.g. on stackoverflow.com/questions/40263117/… the CV resp. the fitting is done within a for loop. How does Keras know in "model.fit()" which folds it has to use when it is not done inside a loop? Then: using "validation_set" instead of a separate data set is recommendable? And "model.evaluate()" is not necessary in this case, right?
– Ben
Oct 9 '19 at 6:29
• Keras basically knows it by 'for loop'. I think the code in link you mentioned is not correct. When you use K-fold cross-validation, K-1 folds will be used for training and some portion can be used for validation. The last fold should be used for testing. Oct 11 '19 at 8:06
• Thanks, so this means I have to fit the model within a for-loop?
– Ben
Oct 11 '19 at 8:46
• You can use train-test- validation split (say 60% train, 20% validation and 20% test.) when you have large dataset. In that case you dont need to use for loop. For smaller dataset normally K-fold is used and in that case for loop can be applied. If you find this answer satisfactory, please don forget to mark right. Thank you Oct 11 '19 at 16:32
• Yes. train_test_split(), you should not use a loop because you have one set of training and one set of testing data. But in Kfold(), say you use k = 5 that is means 5 fold cross-validation. In this case, you will get 5 equal set of data from where you want to use 4 set as tarining and the rest for tseting. Iterate this process for other set also. towardsdatascience.com/…. This post will help you to understand the cross-validation technique Oct 15 '19 at 22:57