4
$\begingroup$

How could we get feature_importances when we are performing regression with XGBRegressor()?

There is something like XGBClassifier().feature_importances_?

$\endgroup$
  • $\begingroup$ (XGBClassifier().feature_importances_) it is right , where is the problem ?? $\endgroup$ – Abhishek Verma Jun 21 '17 at 16:30
  • $\begingroup$ @Abhishek I can't use feature_importances_ with XGBRegressor(), because it works only with XGBClassifier(). $\endgroup$ – Simone Jun 21 '17 at 16:46
  • $\begingroup$ I had to use: model.get_booster().get_score(importance_type='weight') $\endgroup$ – RyGuy Jun 7 '18 at 22:54
7
$\begingroup$
from xgboost import XGBClassifier
model = XGBClassifier.fit(X,y)

# importance_type = ['weight', 'gain', 'cover', 'total_gain', 'total_cover']
model.get_booster().get_score(importance_type='weight')

However, the method below also returns feature importance's and that have different values to any of the "importance_type" options in the method above. This was raised in this github issue, but there is no answer [as of Jan 2019].

model.feature_importances_
$\endgroup$
  • 1
    $\begingroup$ The values in the list feature_importances_ equal the values in the dict get_score(importance_type='weight') where each element is divided by the sum of elements. $\endgroup$ – Anton Tarasenko Nov 22 '18 at 11:33
  • 1
    $\begingroup$ Which importance_type is equivalent to the sklearn.ensemble.GradientBoostingRegressor version of feature_importances_? My suspicion is total_gain $\endgroup$ – Keith Jan 18 '19 at 18:49
3
$\begingroup$

In the past the Scikit-Learn wrapper XGBRegressor and XGBClassifier should get the feature importance using model.booster().get_score(). Not sure from which version but now in xgboost 0.71 we can access it using

model.feature_importances_
$\endgroup$
  • $\begingroup$ I'm using from xgboost.sklearn import XGBRegressor in version 0.72.1 and this worked for me. Thanks! $\endgroup$ – Adam Jul 20 '18 at 17:57
2
$\begingroup$

Finally I have solved this issue by:

model.booster().get_score(importance_type='weight')

$\endgroup$
  • $\begingroup$ But mine returned an error : TypeError: 'str' object is not callable $\endgroup$ – Tony Wang Mar 20 '18 at 9:39
  • $\begingroup$ A bit off-topic, have you tried github.com/slundberg/shap for feature importance? It looks a bit complicated at first, but it is better than normal feature importance. $\endgroup$ – TwinPenguins Jun 8 '18 at 10:28
  • $\begingroup$ for me: model.get_score(importance_type='weight') $\endgroup$ – Catbuilts Oct 15 '18 at 7:06
  • $\begingroup$ @TonyWang try model.get_booster().get_score(importance_type='weight') instead. $\endgroup$ – Sndn Jan 26 '19 at 6:46
  • $\begingroup$ Sndn's solution worked for me as on 04-Sep-2019 $\endgroup$ – Tanveer Uddin Sep 4 '19 at 4:27

Not the answer you're looking for? Browse other questions tagged or ask your own question.