1
$\begingroup$

I have 8 models which I have trained on 90% of my set (training set) and tracked its performance on the loss of the validation set (10% of the original set). I want to generate an ensemble model by blending the different models with a dense layer. However, the validation set only consists of 64 samples and I am concerned that this might cause severe overfitting. I was wondering whether it would make sense to augment the validation data, so that I can increase the number of instances and help in preventing overfitting.

$\endgroup$
0
$\begingroup$

Have you considered freezing the ensemble models themselves and allowing only the weights in final layer to change? Then the overfitting shouldn't be so pronounced...

You might also want to consider cross validation procedure.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.