1
$\begingroup$

I was training a Logistic Regression model over a fairly large dataset with ~1000 columns.

I did apply scaling of features using MinMaxScaler.

I was wondering how to interpret the coefficients generated by the model and find something like feature importance in a Tree based model.

Should I re-scale the coefficients back to original scale to interpret the model properly?

It will be great if someone can shed some light on how to interpret the Logistic Regression coefficients correctly.

$\endgroup$
1
$\begingroup$

No, you do not need to re-scale the coefficients. To the contrary - if they are scaled, you can use them as a way to compare feature importance.

Let's assume that our logistic regression model has coefficients {$ a_i$}, relating to the different (scaled) variables {$x_i$}.
A change of $\Delta x_i $ in the variable $ x_i $ will result in an increase (or decrease, if $a_i$ is negative) of $ a_i \Delta x_i $ in $ log({\hat p_i \over {1-\hat p_i}}) $, i.e. the logit function of $ \hat p_i $, where $ \hat p_i $ is the predicted probability that the i-th example is in the positive class.

So, if the variables are scaled, you can say that if $ a_i$ is larger, then $x_i$ is more important in the model.

$\endgroup$
2
  • $\begingroup$ How to interpret the negative coefficients? $\endgroup$ – Sreeram TP Nov 18 '19 at 4:21
  • $\begingroup$ In the same way, but in the opposite direction (edited answer accordingly) $\endgroup$ – Itamar Mushkin Nov 20 '19 at 7:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.