Could changing the hyperparameters of a model change relative feature importance?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Yes. The most obvious example is when using a Lasso regression : for an increasing $\alpha$ parameter you will have more and more coefficients set to zero. This resulting in a smaller set of features and thus a bigger share of feature importance for remaining features.
-
$\begingroup$ Right but could relative feature importance change? Like say "longitude" was ranked better than "latitude" for a given set of parameters. Could this be reversed (so "latitude" ranked better) for some other set of parameters? $\endgroup$ Apr 6, 2020 at 23:22
-
1$\begingroup$ Yes. Features having similar importance switch easily when changing parameters. (even when taking a different sample of data) $\endgroup$– RusoibaApr 11, 2020 at 20:11