6
$\begingroup$

I am building a standard RandomForest classifier (named model, see the code below) using scikit-learn package. Now, I want to get all parameters of one Randomforest classifier (including its trees (estimators)), so that I can manually draw the flow chart for each tree of the RandomForest classifier. I wonder if anyone knows how it can be done?

Thank you in advance.

Manh

#Import Library
from sklearn.ensemble import RandomForestClassifier #use RandomForestRegressor for regression problem
#Assumed you have, X (predictor) and Y (target) for training data set and x_test(predictor) of test_dataset
# Create Random Forest object
model= RandomForestClassifier(n_estimators=10, max_depth=5) #n_estimators=1000 oob_score = True
#====
#X, y = input_X, input_y

from sklearn.cross_validation import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.2, random_state = 4)

# Train the model using the training sets and check score
model.fit(X_train, y_train)

#Predict Output
y_pred_train = model.predict(X_train)
y_pred_test = model.predict(X_test)

#accuracy
from sklearn.metrics import accuracy_score
print(accuracy_score(y_train,y_pred_train))
print(accuracy_score(y_test,y_pred_test))
$\endgroup$
4
  • 3
    $\begingroup$ RandomForestClassifier has an attribute estimators_ that is a list of all the sub trees. You can use that to inspect the individual trees. You can follow this example from the documentation. $\endgroup$
    – oW_
    Jun 19 '17 at 19:47
  • $\begingroup$ Thanks oW_21, I did see the decision_path attribute of Randomforest classifier. I have been playing around with decision_path and realized that this is rather quite complicated. At the same time, I notice that "estimators_" is not available in my code. The following error is shown "AttributeError: 'RandomForestClassifier' object has no attribute 'estimator_'". Thank you for the code of Decision Tree - It will help alot. $\endgroup$
    – max33587
    Jun 19 '17 at 20:26
  • $\begingroup$ maybe you're missing an s... model.estimators_[0].tree_ should give you the first tree in the forest $\endgroup$
    – oW_
    Jun 19 '17 at 20:38
  • 1
    $\begingroup$ Many thanks! Overall, I think that the retrieval process of the tree structure is not straight forward and I will need to spend more time on the example that you posted to understand it. $\endgroup$
    – max33587
    Jun 19 '17 at 20:49
2
$\begingroup$

I think Terence Parr's answer is now partly outdated. You can get the same (and more) with:

print("Tree depths: ", [t.get_depth() for t in model.estimators_])
print("Tree number of leaves: ", [t.get_n_leaves() for t in model.estimators_])
$\endgroup$
1
$\begingroup$

Max depth is a pretty useful metric, which I didn't find in the API so I wrote this:

def dectree_max_depth(tree):
    n_nodes = tree.node_count
    children_left = tree.children_left
    children_right = tree.children_right

    def walk(node_id):
        if (children_left[node_id] != children_right[node_id]):
            left_max = 1 + walk(children_left[node_id])
            right_max = 1 + walk(children_right[node_id])
            return max(left_max, right_max)
        else: # leaf
            return 1

    root_node_id = 0
    return walk(root_node_id)

You can use it on all trees in a forest (rf) like this:

[dectree_max_depth(t.tree_) for t in rf.estimators_]
$\endgroup$
1
$\begingroup$

You can select and visualization individual trees from a Random Forest:

# Extract individual tree from forest
tree_id = 5
tree = model.estimators_[tree_id]

# Draw individual tree flowchart
from sklearn.tree import export_graphviz

export_graphviz(tree)
$\endgroup$
1
  • $\begingroup$ In a jupyter notebook, you would do: import graphviz; graphviz.Source(export_graphviz(tree)) $\endgroup$
    – user650654
    Sep 2 '20 at 0:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.