3
$\begingroup$

Based on answers to this question, I should be able to build a random forest with all the same trees by using bootstrap = False, max_features = None, random_state = 42 parameters.

I wrote quick code to test it, and it seems that different trees are created.

Is it possible to create a random forest using RandomForestClassifier which will produce the same trees?

$\endgroup$
6
  • $\begingroup$ Why do you want them all to be the same? $\endgroup$
    – kbrose
    Jul 5, 2018 at 13:45
  • $\begingroup$ it's right there in the accepted answer to the linked question if you read the entire answer $\endgroup$
    – oW_
    Jul 5, 2018 at 15:27
  • 1
    $\begingroup$ @kbrose - as an example step in process of creating decision tree, looking for its hyperparameters, creating homogenous forest (with those hyperparameters), adding bootstrap, adding fraction of features and increasing n_estimators. So such forest would be something different than single tree, but with exactly the same predictive capability. $\endgroup$ Jul 6, 2018 at 5:48
  • $\begingroup$ @oW_ Yes, I read the entire answer, especially You can fix this by passing the same seed to each DecisionTreeClassifier which you can do using random_state=something. RandomForestClassifier also has a random_state parameter which it passes along each DecisionTreeClassifier. - I did that, and it did produced trees with fixed seed but different for each one. So it seems that there is no such possibility? $\endgroup$ Jul 6, 2018 at 5:51
  • 1
    $\begingroup$ this homogenous forest would not be different from a single decision tree. If they all share the same random seed then the randomness introduced by bootstrap and max_features would disappear ... that's exactly the point... that's also the reason why you can't do it with RandomForestClassifier..unless I'm missing something here $\endgroup$
    – oW_
    Jul 6, 2018 at 15:22

1 Answer 1

4
$\begingroup$

You want to create a random forest where all decision trees are the same. (I am trusting you are trying to do this as an educational exercise, not as something serious. :))

You tried to do this by setting random_state and cited my answer from

Indeed, as you discovered, my answer is not fully correct.

I have edited it to add

While this removes the stochasticity component of the training, the decision trees would still be different. The thing is that sklearn ensembles generate a new random seed for each child based on the random state they are given. They do not pass along the same random_state.

You can see this is the case by checking the _set_random_states method from the ensemble base module, in particular this line, which propagates the random_state across the ensembles' children.

As you can see, sklearn uses the random_state you give RandomForest to create new random states for each child,

for key in sorted(estimator.get_params(deep=True)):
    if key == 'random_state' or key.endswith('__random_state'):
        to_set[key] = random_state.randint(MAX_RAND_SEED)

Therefore they will be different. This is done presumably because heterogeneity is what makes an ensemble powerful. Notice this is not specific to RandomForest, but to any sklearn ensemble.

Sorry for my incorrect answer to the question you cite. If you want, you can use the random forest implementation. That one is completely homogeneous.

$\endgroup$
1
  • $\begingroup$ Thanks very much for correction and additional information. Thats the information I was looking for! $\endgroup$ Jul 7, 2018 at 6:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.