I have a classification dataset with 10k instances and 4 classes and it is unbalanced. 7000 of it belongs to first class, 2000 of it belongs to second 800 of it belongs to third class and remaining instances are belong to fourth class. I have a single neural network based on MLP.
At the end of down-sampling process, I am planning to have 2000,2000,800,200 instances. (I can even further down-sample first 3 classes until I have 200 instances from each class but I don't want to decrease size of the dataset that much).
My question is: If I train my MLP for N epoch, should I use the same subset for class-1 for all of the epochs or should I generate this subset randomly for each epoch?