Spliting Training Test and Validation for Image Dataset

Question

I have 600 images in the training folder, 200 images in the validation folder, and 200 images in the test folder. Suppose if I fit the training data generator and validation data generator for some epochs for learning purposes - model.fit(train,val), and after that, I add both the training and validation data like 600 + 200 = 800 and for those 800 images I fit the new test data set which consists of 200 images and find the accuracy for this. Is this good practice to get a better model performance?

I am new to deep learning, your answers will be very much helpful to gain some insights about the splitting of data.

Answer 1

It's bad practice to train a model and not have an independent way to evaluate its suitability or performance relative to a metric.

It's tempting to think that adding more data produces a better model but that is not the case as you may have to make adjustments to it you didn't anticipate.

Also, if you didn't split the data sets then you have the added risk that one or more of the datasets are not representative of the others. But having a test and validation set you could determine this.

Answer 2

Combining training and validation data as training data before evaluating test data should be okay. Think of the step that uses validation data as part of the training process, then you will see that it is okay.

The purpose is to get an accurate estimate of how the model will perform on unseen data. If you can afford it, a better estimate can be obtained by doing k-fold cross-validation, e.g. for k = 5:

• Divide all available data into 5 folds (200 images in each)
• For each fold, do the following:
  • Treat that fold as the test set
  • Split the remaining data (800 images) into training and development/validation data
  • Complete the training process and measure performance on the test set
• Average the performance across the 5 folds to get a better estimate of performance on unseen data.

Answer 3

K-fold validation may be used to take advantage of 800 images during the train/validation stages. You should keep the testing data completely separate and use it only for the final metrics / scoring stage.