Could changing the hyperparameters of a model change relative feature importance?


1 Answer 1


Yes. The most obvious example is when using a Lasso regression : for an increasing $\alpha$ parameter you will have more and more coefficients set to zero. This resulting in a smaller set of features and thus a bigger share of feature importance for remaining features.

  • $\begingroup$ Right but could relative feature importance change? Like say "longitude" was ranked better than "latitude" for a given set of parameters. Could this be reversed (so "latitude" ranked better) for some other set of parameters? $\endgroup$ Commented Apr 6, 2020 at 23:22
  • 1
    $\begingroup$ Yes. Features having similar importance switch easily when changing parameters. (even when taking a different sample of data) $\endgroup$
    – Rusoiba
    Commented Apr 11, 2020 at 20:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.