I'm fitting a neural network using a very small data set, so try splitting the data into training and validation sets. (there is a separate test set) If I split training/validation randomly multiple times, construct a neural network for each training/validation split, and take average of predicted values of the neural networks on the test set, can it be called a ensemble model? Or is there a specific name for such a technique?

Edit: I just found that a similar technique is called 'repeated random sub-sampling validation,' but the RRSSV splits the data into training and test set (though it is called 'validation data' according to Wikipedia, it's actually test data). My method splits the given data into training and validation set, and use separate test data. I think my method can also be called RRSSV.


I think even this method is also called Ensemble Method.

How could I conclude that?

  • You might have heard about this algorithm named Random Forest, what does it do? It take data randomly at row level and column level builds different trees and takes an average of it. It is also considered as one of the best algorithm for Prediction and Classification. Can go through this explanation for better understanding. Random Forest is called Ensemble model(of trees).

A suggestion, as you have mentioned in the question that you have very less data, at that time the models cannot generalize well and you cannot achieve good results. If you have any way to increase the dataset size by collecting more data, it could aid you in achieving better accuracy. This is also explained the link attached(explanation). Do go through and let me know if you have any additional questions.


You might try extending your approach to include adding random noise to your training data, sometimes called noise injection. By doing this you can theoretically extend the amount of training data you have almost infinitely and avoid over-fitting of a small training sample. An internet search will turn up several papers on the subject, e.g. Whiteout: Gaussian Adaptive Noise Regularization in FeedForward Neural Networks


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.