4
$\begingroup$
>>> from sklearn.ensemble import RandomForestClassifier

>>> clf = RandomForestClassifier(n_estimators=10, random_state=1)
>>> Y=[0,1]

>>> X = [[3,2,1,0], [7,6,5,4]]
>>> clf = clf.fit(X, Y)
>>> print clf.feature_importances_
[ 0.2  0.1  0.1  0. ]

>>> X = [[0, 1, 2,3], [4,5,6,7]]
>>> clf = clf.fit(X, Y)
>>> print clf.feature_importances_
[ 0.2  0.1  0.1  0. ]

>>> X = [[3,2,1,0], [7,6,5,4]]
>>> clf = clf.fit(X, Y)
>>> print clf.feature_importances_
[ 0.2  0.1  0.1  0. ]

>>> X = [[3,1,2,0], [7,5,6,4]]
>>> clf = clf.fit(X, Y)
>>> print clf.feature_importances_
[ 0.2  0.1  0.1  0. ]

Assume the features have names. When I shuffle/change the listed order of features specified in the training data set, the importance for each feature changes.
That means the resulted random forest classifier also changes. Note that I have rule out the effect of randomness, by fixing the random seed.

Why does the listed order of features specified in the data set matter to the random forest classifier, given that the random seed is fixed? Thanks.

$\endgroup$
5
$\begingroup$

Since your fixed the seed, with default option bootstrap=True each tree is build on a fixed subsample of your data. With the default option max_features='auto' each tree uses 2 features. Since your fixed the seed, it is always the same, let us say, "1st and 4th" features for the first tree. And they are different in each of your shuffles. So, the tree is different. The same applies to each tree.

By the way, with 4 features in total, only 6 (4*3/2=6) various trees are possible. With n_estimators=10 some of these trees are necessarily repeated. And in each shuffle those are different trees.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.