My goal is to extract the feature importances from already trained random forest classifier and transfer them to another classifier. How this can be done? and How can access to modify feature_importances of Random Forest Classifier model? using scikit learn.
I'm not sure it's possible to "transfer" the feature importances from model to model in a Random Forest Classifier. Although I think there are two work-arounds you may be able to use.
The first option you have is to retrain the second model on the exact same training data. This will have the effect of both models having close to the same feature importances, though I think doesn't guarantee they will be exactly the same.
Another way would be to try knowledge distillation (https://arxiv.org/abs/1503.02531). Here you're basically making a target, or student, model train to not only match the hard labels, but also a set of soft labels, generated from a teacher network. In this case, the model that you want to transfer the knowledge to would be the student network. However, I believe this will involve writing a custom loss function, which I don't think is possible in sklearn. The paper I linked is in the context of neural networks so I also don't know if there would be any gaps in the performance between model types, as presumably, you would doing this with a random forest.
Just to confirm, I ran this short script that trains two models on the same data and it seems to be doing what you want it to:
from sklearn.ensemble import RandomForestClassifier from sklearn.datasets import make_classification X, y = make_classification(n_samples=1000, n_features=4, n_informative=2, n_redundant=0, random_state=0, shuffle=False) clf = RandomForestClassifier(max_depth=2, random_state=0) clf.fit(X, y) print('model 1 feature importances: ', clf.feature_importances_) clf2 = RandomForestClassifier(max_depth=2, random_state=0) clf2.fit(X, y) print('model 2 feature importances: ', clf2.feature_importances_)
The results I got were:
model 1 feature importances: [0.14205973 0.76664038 0.0282433 0.06305659] model 2 feature importances: [0.14205973 0.76664038 0.0282433 0.06305659]
Here you can see that the results were the exact same. I would imagine that the integrity of the transfer may change as the data changes. Make sure both models are initialized with the same random state if you want the feature importance values to match exactly.