I've trained a gradient boosting classification model. But, suppose i've a set of fixed features F1,F2....Fn and new features which are added weekly (no. of actions done in that week). So, after 2 weeks dataset to be trained on is :

   Fixed           Dynamic
F1 ,F2 .....Fn    W1 ,W2

After 3 weeks

  Fixed           Dynamic
F1 ,F2 .....Fn    W1 ,W2, W3

How do we approach this problem on production server, is there any approach available which allow model to be retrained on new features and not only on new observations ?

  • 2
    $\begingroup$ Pls elaborate about these new features w1, w2, ... that become available over time. I suspect they're not really indep features that require a new column in your design matrix. $\endgroup$
    – horaceT
    Mar 3 '18 at 19:29

The way gradient boosting is constructed and trained, there is not an obvious solution for this without just training from scratch. Other models might be more suitable for this (Adding this to a neural network and retraining this will take less time than from scratch I think). Another approach would be to use these time-based lag features as a time series, that way you just input your features and it will learn relationships between the past few weeks instead.

  • $\begingroup$ Thanks jan van der vegt ! I guess MLP will be better then. But, i'm still new to retraining models. It'll be very helpful if you can point me to some resource regarding this. Also, can time series be used along with those fixed features ? $\endgroup$
    Sep 4 '17 at 9:21
  • $\begingroup$ You could use a fixed number of lags as features to keep them fixed, for the following week just shift them by a week. With regards to retraining models, I don't know any specific resources, this is more just experience. With MLP you would keep the weights from the previous model, add a node at the input and the additional weights and then retrain on all your data. I think just retraining from scratch is by far the easiest solution if you are a bit new $\endgroup$ Sep 4 '17 at 12:14
  • $\begingroup$ @JanvanderVegt This question has nothing to do with model choice. One faces the same problem even in a linear model, and that is, how to incorporate a new feature which did not exist before time t. $\endgroup$
    – horaceT
    Mar 3 '18 at 19:34
  • $\begingroup$ Of course, and some models lend themselves better for adding features to an already trained model than others. It's similar to semi-supervised learning in that regard, except that you have a missing feature in the past instead of missing targets. $\endgroup$ Mar 3 '18 at 20:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.