15
$\begingroup$

A Random Forest (RF) is created by an ensemble of Decision Trees's (DT). By using bagging, each DT is trained in a different data subset. Hence, is there any way of implementing an on-line random forest by adding more decision tress on new data?

For example, we have 10K samples and train 10 DT's. Then we get 1K samples, and instead of training again the full RF, we add a new DT. The prediction is done now by the Bayesian average of 10+1 DT's.

In addition, if we keep all the previous data, the new DT's can be trained mainly in the new data, where the probability of picking a sample is weighted depending how many times have been already picked.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
9
$\begingroup$

There's a recent paper on this subject (On-line Random Forests), coming from computer vision. Here's an implementation, and a presentation: Online random forests in 10 minutes

Improve this answer
$\endgroup$
5
  • $\begingroup$ The implementation that you mentioned follows a tree-growing strategy, like Mondrian forests (arxiv.org/abs/1406.2673). Hence, the number of trees is constant while the number of splits is increased. My question focuses on increasing the number of trees for new samples while remaining untouched the previously trained trees. $\endgroup$
    – tashuhka
    Oct 24, 2014 at 9:20
  • 1
    $\begingroup$ Like this? Don't you also want to drop trees if appropriate? $\endgroup$
    – Emre
    Oct 24, 2014 at 16:31
  • $\begingroup$ Thank you. This is more similar to what I am looking for. In this case, the use RF for feature selection of time-variant signals. However, the specific implementation and validity of the method is quite unclear, do you know if they published anything (Google didn't help)? $\endgroup$
    – tashuhka
    Oct 30, 2014 at 13:59
  • 1
    $\begingroup$ Calculating Feature Importance in Data Streams With Concept Drift Using Online Random Forest $\endgroup$
    – Emre
    Oct 30, 2014 at 18:01
  • $\begingroup$ Thanks for the link! I can see that they actually update all the previous trees using a tree-growing strategy, and I am interested in creating new DT's with the new data while keeping untouched the old trees. $\endgroup$
    – tashuhka
    Oct 31, 2014 at 10:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.