1
$\begingroup$

I was training a Logistic Regression model over a fairly large dataset with ~1000 columns.

I did apply scaling of features using MinMaxScaler.

I was wondering how to interpret the coefficients generated by the model and find something like feature importance in a Tree based model.

Should I re-scale the coefficients back to original scale to interpret the model properly?

It will be great if someone can shed some light on how to interpret the Logistic Regression coefficients correctly.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

No, you do not need to re-scale the coefficients. To the contrary - if they are scaled, you can use them as a way to compare feature importance.

Let's assume that our logistic regression model has coefficients {$ a_i$}, relating to the different (scaled) variables {$x_i$}.
A change of $\Delta x_i $ in the variable $ x_i $ will result in an increase (or decrease, if $a_i$ is negative) of $ a_i \Delta x_i $ in $ log({\hat p_i \over {1-\hat p_i}}) $, i.e. the logit function of $ \hat p_i $, where $ \hat p_i $ is the predicted probability that the i-th example is in the positive class.

So, if the variables are scaled, you can say that if $ a_i$ is larger, then $x_i$ is more important in the model.

Improve this answer
$\endgroup$
2
  • $\begingroup$ How to interpret the negative coefficients? $\endgroup$
    – Sreeram TP
    Commented Nov 18, 2019 at 4:21
  • $\begingroup$ In the same way, but in the opposite direction (edited answer accordingly) $\endgroup$
    – Itamar Mushkin
    Commented Nov 20, 2019 at 7:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.