How could we get feature_importances when we are performing regression with XGBRegressor()?
There is something like XGBClassifier().feature_importances_?
$\begingroup$
$\endgroup$
3
3 Answers
$\begingroup$
$\endgroup$
2
from xgboost import XGBClassifier
model = XGBClassifier.fit(X,y)
# importance_type = ['weight', 'gain', 'cover', 'total_gain', 'total_cover']
model.get_booster().get_score(importance_type='weight')
However, the method below also returns feature importance's and that have different values to any of the "importance_type" options in the method above. This was raised in this github issue, but there is no answer [as of Jan 2019].
model.feature_importances_
-
2$\begingroup$ The values in the list
feature_importances_equal the values in the dictget_score(importance_type='weight')where each element is divided by the sum of elements. $\endgroup$ Nov 22, 2018 at 11:33 -
2$\begingroup$ Which importance_type is equivalent to the sklearn.ensemble.GradientBoostingRegressor version of feature_importances_? My suspicion is total_gain $\endgroup$– KeithJan 18, 2019 at 18:49
$\begingroup$
$\endgroup$
5
Finally I have solved this issue by:
model.booster().get_score(importance_type='weight')
-
1$\begingroup$ But mine returned an error : TypeError: 'str' object is not callable $\endgroup$ Mar 20, 2018 at 9:39
-
$\begingroup$ A bit off-topic, have you tried github.com/slundberg/shap for feature importance? It looks a bit complicated at first, but it is better than normal feature importance. $\endgroup$ Jun 8, 2018 at 10:28
-
$\begingroup$ for me:
model.get_score(importance_type='weight')$\endgroup$ Oct 15, 2018 at 7:06 -
$\begingroup$ @TonyWang try
model.get_booster().get_score(importance_type='weight')instead. $\endgroup$– SndnJan 26, 2019 at 6:46 -
$\begingroup$ Sndn's solution worked for me as on 04-Sep-2019 $\endgroup$ Sep 4, 2019 at 4:27
$\begingroup$
$\endgroup$
1
In the past the Scikit-Learn wrapper XGBRegressor and XGBClassifier should get the feature importance using model.booster().get_score(). Not sure from which version but now in xgboost 0.71 we can access it using
model.feature_importances_
-
$\begingroup$ I'm using
from xgboost.sklearn import XGBRegressorin version 0.72.1 and this worked for me. Thanks! $\endgroup$– AdamJul 20, 2018 at 17:57
feature_importances_withXGBRegressor(), because it works only withXGBClassifier(). $\endgroup$