2
$\begingroup$

I have searched a lot on the web for this question, but I never seem to find a consistent yet straight forward answer. Simply put, the question is: How exactly does scaling affect logistic regression? What should you expect from a logistic regression classifier when the data is scaled and when it is not? Is there any difference between min-max scaling and standard scaling in terms of logistic regression?

$\endgroup$
4
$\begingroup$

It affects anything optimized by a form of gradient descent, because it affects the relative scale of the dimensions of the input. If A is generally 1000x larger than B, then changing B's coefficient by some amount is in a sense a 1000x bigger move. In theory this won't matter but in practice it can cause the gradient descent to have trouble landing in the right place in B's dimension.

I think the more significant effect may be regularization. All terms are penalized equally in a simple formulation of regularization. Again if A is generally 1000x larger than B, its coefficient will tend be 1000x smaller (all else equal) than B's, so will be far less penalized, when there's no particular reason to treat them differently.

Finally it affects interpretation, if you want to view the coefficients' magnitudes as correlating with importance, for the same reason.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.