I am trying to build a majority vote system for 3 Neural Networks, and I came across the concept of Bagging method. Actually, I want to use neural networks as weak learners (I know it's debatable, but some papers have tried it and I want to try it too).
For more information about the voting system I tried to construct/constructed, please read the following thread (The softmax Layer is better the the function in the thread, because the majority function only gives equivalent accuracies of the 3 NNs, but doesn't improve the overall accuracy).
I read that bagging can improve the overall accuracy of the weak learners, but as you can see I only have 3 learners and there aren't any clear information about bagging with Neural Networks. I only found after some reading that I can use ensemble learning with Neural networks by using the output of the trained NNs in a linear fashion. If I want to detail this into steps, I would write:
- Divide Dataset into training and validation sets
- From the Training sat, construct 3 bootstraps samples ?, Or I need more?
- Train/Develop the 3 neural Networks on the 3 bootstraps samples?
- Test the Neural Networks on the validation/test set
- How can I join decisions of the NNs by bagging?
If possible, I need your insight on these implementation steps and to know if they follow the best practices of bagging with NNs.
PS: I just started reading about bagging and boosting, so I apologize for any conceptual mistakes and contradictions I might have said.