A Random Forest (RF) is created by an ensemble of Decision Trees's (DT). By using bagging, each DT is trained in a different data subset. Hence, is there any way of implementing an on-line random forest by adding more decision tress on new data?

For example, we have 10K samples and train 10 DT's. Then we get 1K samples, and instead of training again the full RF, we add a new DT. The prediction is done now by the Bayesian average of 10+1 DT's.

In addition, if we keep all the previous data, the new DT's can be trained mainly in the new data, where the probability of picking a sample is weighted depending how many times have been already picked.


There's a recent paper on this subject (On-line Random Forests), coming from computer vision. Here's an implementation, and a presentation: Online random forests in 10 minutes

| improve this answer | |
  • $\begingroup$ The implementation that you mentioned follows a tree-growing strategy, like Mondrian forests (arxiv.org/abs/1406.2673). Hence, the number of trees is constant while the number of splits is increased. My question focuses on increasing the number of trees for new samples while remaining untouched the previously trained trees. $\endgroup$ – tashuhka Oct 24 '14 at 9:20
  • 1
    $\begingroup$ Like this? Don't you also want to drop trees if appropriate? $\endgroup$ – Emre Oct 24 '14 at 16:31
  • $\begingroup$ Thank you. This is more similar to what I am looking for. In this case, the use RF for feature selection of time-variant signals. However, the specific implementation and validity of the method is quite unclear, do you know if they published anything (Google didn't help)? $\endgroup$ – tashuhka Oct 30 '14 at 13:59
  • 1
  • $\begingroup$ Thanks for the link! I can see that they actually update all the previous trees using a tree-growing strategy, and I am interested in creating new DT's with the new data while keeping untouched the old trees. $\endgroup$ – tashuhka Oct 31 '14 at 10:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.