1
$\begingroup$

I read that L2 regularization in logistic regression creates a sort of sphere that limits the choice of $w$ weight, but why does this happen?

$\endgroup$
5
$\begingroup$

Your question is really about the method of Lagrange multipliers in constrained optimization, not logistic regression per se. The gist of it is that a constrained optimization problem can be recast as an unconstrained optimization problem by adding a term, called the regularizer, and vice versa. The sphere comes from recasting the unconstrained problem into a constrained one; recall that a constant $L_2$ norm defines a hypersphere.

$\endgroup$
1
$\begingroup$

A simple way to think about this is to appreciate that you are minimizing an objective function. L2 regularization alters the output of the objective function such that smaller values are favored. So you have this constant 'pressure' on the parameters aiming towards 0.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.