Augmenting the validation set in Ensemble Model

Question

I have 8 models which I have trained on 90% of my set (training set) and tracked its performance on the loss of the validation set (10% of the original set). I want to generate an ensemble model by blending the different models with a dense layer. However, the validation set only consists of 64 samples and I am concerned that this might cause severe overfitting. I was wondering whether it would make sense to augment the validation data, so that I can increase the number of instances and help in preventing overfitting.

Answer 1

Have you considered freezing the ensemble models themselves and allowing only the weights in final layer to change? Then the overfitting shouldn't be so pronounced...

You might also want to consider cross validation procedure.