I read that L2 regularization in logistic regression creates a sort of sphere that limits the choice of $w$ weight, but why does this happen?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Your question is really about the method of Lagrange multipliers in constrained optimization, not logistic regression per se. The gist of it is that a constrained optimization problem can be recast as an unconstrained optimization problem by adding a term, called the regularizer, and vice versa. The sphere comes from recasting the unconstrained problem into a constrained one; recall that a constant $L_2$ norm defines a hypersphere.
$\begingroup$
$\endgroup$
A simple way to think about this is to appreciate that you are minimizing an objective function. L2 regularization alters the output of the objective function such that smaller values are favored. So you have this constant 'pressure' on the parameters aiming towards 0.
