1
$\begingroup$

Hello I am using keras to develop a neural network model and I have a data of 45 numerical predictor variables, 2 categorical targets that will be predicted each with a different model. As I found, there is no feature importance model in keras. There are three options I can use, correlation ratio between the variables, kendals rank coefficient values and lasso regulation. Which one do you think is suitable to be used for feature selection in neural networks?

$\endgroup$
2
$\begingroup$

Unfortunately, there is no direct way to assess the "importance" of a variable in a Neural Network. One option, very time consuming, consists in removing each variable, one by one, replacing it with random noise, and checking how the performance changes. That will give you an idea on the contribution of a variable.

Alternatively, stick with importance scores of Tree-based models (such as Random Forests), or with good old statistical analysis. Shapley value regressions are a famous example.

$\endgroup$
3
  • $\begingroup$ I think I will be using correlation ratio and sort the variables accordingly. Any other suggestions? $\endgroup$
    – Jean
    May 22 '20 at 19:22
  • $\begingroup$ Why is random noise better than setting the feature (or associated weights) to 0? $\endgroup$
    – xi45
    May 26 '20 at 12:25
  • $\begingroup$ Substituting the actual variable with random noise is just a way to "shut down" that variable for that specific session. In that way you can counterfactually assess the importance of the missing variable. Noise by definition cannot be learned. I can't say if it's better or worse than your option, that is task-specific I guess. $\endgroup$
    – Leevo
    May 26 '20 at 13:53
2
$\begingroup$

For feature importance, you might want to consider using Shapley values or LIME. There are some examples here.

$\endgroup$
1
  • $\begingroup$ Thank you Thomas. I will take a look. $\endgroup$
    – Jean
    May 22 '20 at 19:23
0
$\begingroup$

To complement the other two answers, you can use various model-agnostic methods to assess feature importance. See this nice e-book: Interpretable Machine Learning - A Guide for Making Black Box Models Explainable by Christoph Molna

$\endgroup$
1
  • $\begingroup$ Thank you, I am looking into it $\endgroup$
    – Jean
    May 26 '20 at 22:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.