# Xgboost - How to use feature_importances_ with XGBRegressor()?

How could we get feature_importances when we are performing regression with XGBRegressor()?

There is something like XGBClassifier().feature_importances_?

• (XGBClassifier().feature_importances_) it is right , where is the problem ?? Jun 21 '17 at 16:30
• @Abhishek I can't use feature_importances_ with XGBRegressor(), because it works only with XGBClassifier(). Jun 21 '17 at 16:46
• I had to use: model.get_booster().get_score(importance_type='weight') Jun 7 '18 at 22:54

from xgboost import XGBClassifier
model = XGBClassifier.fit(X,y)

# importance_type = ['weight', 'gain', 'cover', 'total_gain', 'total_cover']
model.get_booster().get_score(importance_type='weight')


However, the method below also returns feature importance's and that have different values to any of the "importance_type" options in the method above. This was raised in this github issue, but there is no answer [as of Jan 2019].

model.feature_importances_

• The values in the list feature_importances_ equal the values in the dict get_score(importance_type='weight') where each element is divided by the sum of elements. Nov 22 '18 at 11:33
• Which importance_type is equivalent to the sklearn.ensemble.GradientBoostingRegressor version of feature_importances_? My suspicion is total_gain Jan 18 '19 at 18:49

In the past the Scikit-Learn wrapper XGBRegressor and XGBClassifier should get the feature importance using model.booster().get_score(). Not sure from which version but now in xgboost 0.71 we can access it using

model.feature_importances_

• I'm using from xgboost.sklearn import XGBRegressor in version 0.72.1 and this worked for me. Thanks!
Jul 20 '18 at 17:57

Finally I have solved this issue by:

model.booster().get_score(importance_type='weight')

• But mine returned an error : TypeError: 'str' object is not callable Mar 20 '18 at 9:39
• A bit off-topic, have you tried github.com/slundberg/shap for feature importance? It looks a bit complicated at first, but it is better than normal feature importance. Jun 8 '18 at 10:28
• for me: model.get_score(importance_type='weight') Oct 15 '18 at 7:06
• @TonyWang try model.get_booster().get_score(importance_type='weight') instead.
– Sndn
Jan 26 '19 at 6:46
• Sndn's solution worked for me as on 04-Sep-2019 Sep 4 '19 at 4:27