How could we get feature_importances when we are performing regression with XGBRegressor()?

There is something like XGBClassifier().feature_importances_?

  • $\begingroup$ (XGBClassifier().feature_importances_) it is right , where is the problem ?? $\endgroup$ Jun 21, 2017 at 16:30
  • $\begingroup$ @Abhishek I can't use feature_importances_ with XGBRegressor(), because it works only with XGBClassifier(). $\endgroup$
    – Simone
    Jun 21, 2017 at 16:46
  • $\begingroup$ I had to use: model.get_booster().get_score(importance_type='weight') $\endgroup$
    – RyGuy
    Jun 7, 2018 at 22:54
  • $\begingroup$ Yes, the feature_importances_ property is available on XGBClassifier and XGBRegressor: xgboosting.com/xgboost-feature_importances_-property $\endgroup$
    – jasonb
    2 days ago

3 Answers 3

from xgboost import XGBClassifier
model = XGBClassifier.fit(X,y)

# importance_type = ['weight', 'gain', 'cover', 'total_gain', 'total_cover']

However, the method below also returns feature importance's and that have different values to any of the "importance_type" options in the method above. This was raised in this github issue, but there is no answer [as of Jan 2019].

  • 2
    $\begingroup$ The values in the list feature_importances_ equal the values in the dict get_score(importance_type='weight') where each element is divided by the sum of elements. $\endgroup$ Nov 22, 2018 at 11:33
  • 2
    $\begingroup$ Which importance_type is equivalent to the sklearn.ensemble.GradientBoostingRegressor version of feature_importances_? My suspicion is total_gain $\endgroup$
    – Keith
    Jan 18, 2019 at 18:49

Finally I have solved this issue by:


  • 1
    $\begingroup$ But mine returned an error : TypeError: 'str' object is not callable $\endgroup$
    – Tony Wang
    Mar 20, 2018 at 9:39
  • $\begingroup$ A bit off-topic, have you tried github.com/slundberg/shap for feature importance? It looks a bit complicated at first, but it is better than normal feature importance. $\endgroup$ Jun 8, 2018 at 10:28
  • $\begingroup$ for me: model.get_score(importance_type='weight') $\endgroup$
    – Catbuilts
    Oct 15, 2018 at 7:06
  • $\begingroup$ @TonyWang try model.get_booster().get_score(importance_type='weight') instead. $\endgroup$
    – Sndn
    Jan 26, 2019 at 6:46
  • $\begingroup$ Sndn's solution worked for me as on 04-Sep-2019 $\endgroup$ Sep 4, 2019 at 4:27

In the past the Scikit-Learn wrapper XGBRegressor and XGBClassifier should get the feature importance using model.booster().get_score(). Not sure from which version but now in xgboost 0.71 we can access it using

  • $\begingroup$ I'm using from xgboost.sklearn import XGBRegressor in version 0.72.1 and this worked for me. Thanks! $\endgroup$
    – Adam
    Jul 20, 2018 at 17:57

Not the answer you're looking for? Browse other questions tagged or ask your own question.