Sklearn randomforest online learning

Question

Can I do online learning with random forests? I have a few million datapoints and the classifier fails to finish on the cross validation step.

Can i break it up in chunks sequentially?
Current code:

X_train, y_train, X_val, y_val, X_test, y_test = load_dataset()

   print('Planting trees...')
   clf = RandomForestClassifier(
       n_estimators=50,
       max_depth=None,
       min_samples_split=1,
       random_state=0
   )

   print('Growing trees...')
   classifier = clf.fit(X_train, y_train)

   # see how we did
   print('Testing trees...')
   scores = cross_val_score(classifier, X_test, y_test)
   print(scores)
   print('accuracy: %d' % (scores.mean()))

Can I change it to something like:

for chunk in df:
     clf.fit(...)
     cross_validate...

Answer 1

There's nothing out of the box that will do true online learning. In order for a scikit-learn algorithm to support online learning it must provide the partial_fit function, which RandomForestClassifier does not. I think the code you have given will just refit the entire forest on the subset of data it is currently looking at.

One alternative you could try is to initialize RandomForestClassifier with warm_start set to True. Then on each subsequent call to fit, the random forest will add a new set of trees to the ensemble. You may have to reduce the number of estimators in the ensemble because the number of estimators you end up with will be the number you initialized with times the number of chunks you loop over.

This won't behave exactly the same way as if you were to train the entire ensemble in a truly online way, but it may be close enough for your purposes.