I am using sklearn's permutation_importance for feature selection, and all my features return score decreases of 0, even though my model accuracy is 0.96.

I have tried the permutation with other metrics, and only the roc_auc_score actually returns non-zero value, but any other metrics returns zeros for all features. I guess I am confused as to how this is possible, as the feature importance with the following code (should) output the drop in accuracy when we shuffle the values of the different variables.

model = LogisticRegression().fit(X, y)

permutation_score = permutation_importance(model, X, y, scoring = 'accuracy', n_repeats=10) 

On another note, I ran a correlation test between my explanatory variables, and none of the variables seemed too highly correlated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.