# How to train a model using a daunting huge training dataset

I have an extremely huge dataset and I'm wondering me how could be the right way to set an experiment to use this data to train a model.

I understand that I can use data-reduction to, for instance, drop out some variables. In despite data-reduction can actually reduces the data amount, as I can see this technique is intended to improve the model training effectiveness, not to deal with the practical issues that comes out from the data amount.

One of my ideas is to suffle the whole data first and then split the data into 'small' chunks. Once I have, let's say' $$N$$ chuncks, I can train the same model using each chunck as follows:

initialize(M);
for(n in N) {
M = train(M, D);
}


Although this approach can be effective to fit the experiment to computational resources at hand, I'm afraid that training the model this way can affect the model's quality by including bias from the latter chuncks. In addition, N is now a new hyperparameter to be set.

Another alternative I can see is by using statistical sampling:

D = retrieve_sampling(sampling_size);
if (D is good)
M = train(D);


I'm wondering me if there are other ways to do it then the ones I've cited here.

• incremental trainign is an approach that many ML models allow without affecting final performance Jan 27 at 13:00
• Another option would be using partial fit Jan 27 at 15:10