2
$\begingroup$

I have built an ML model with the goal of making predictions for targets of the following week. In general, new data will come in and be processed at the end of each week and be in the same data structure as before.

Instead of re-training the model from scratch for each week's predictions, I am considering applying an incremental learning approach so that past learning is not entirely discarded and the model would (presumably) increase in performance over time. I'm working with sklearn, keras on Python 3.

Couple of questions:

  1. Suppose we performed incremental learning on Dataset 1 and subsequently Dataset 2. If both datasets were actually available, would the performance be exactly the same as if the model was trained on both datasets without any incremental learning? (Assuming no data shuffling)

  2. Are there any advantages to dedicated incremental learning packages such as creme instead of simply saving/loading the models and working with sklearn?

  3. As time increases, there may be drift or pattern changes in the data (i.e. some features may not be as predictive, while others become more so). Is it possible to force the incremental learning model to more heavily weigh the current dataset while discounting some of the learning from the past?
  4. The documentation here is a bit confusing for the actual implementation. When warm_start=True, calling .fit() changes model parameters but data should be "more-or-less constant", whereas calling .partial_fit() keeps model parameters fixed with data being different. My goal is for both model parameters to change (while retaining some past learning) and fitting on new datasets - which one should I be calling?
$\endgroup$