I am new to ML and have been reading online about training bottlenecks when there are frequent updates to data.
Let's say I have a built a model based on a dataset of 10M records.
Now, in another 2 months, I might receive another 1M records which we would like to feed into our model as well.
Similarly this goes on for every 2 months. We would like to update/train our model with latest data as and when it's available
1) But for example, let's say the training takes 1 week for every new data update
2) Any suggestions on how we can minimize the training time (when we train every 2 months?)?
3) Should we select a representative sample from 1 Million datapoints? Is that good enough
4) I understand it's all about tradeoff but I am curious to know whether I am missing any known approaches to save training time? I am thinking representative sample can reduce the sample size and help us fasten the training process
Can you share your suggestions on this?