# How does L1 regularization make low-value features more zero than L2?

Below formulas, L1 and L2 regularization  Many experts said that L1 regularization makes low-value features zero because of constant value. However, I think that L2 regularization could also make zero value. Could you please explain the reason why L1 has more tendency to make zero value? (It's gonna be great if you let me know this reason by using formula(like above equation)!

## 2 Answers

The penalty on coefficients of L1 is more aggressive on values close to zero than it is for L2.

With L1, when a weight value comes closer to zero, it tends to get even closer, because of the $$\epsilon\lambda$$ penalty which stays constant.

With L2, the $$\epsilon\lambda w^{(t)}$$ term gets smaller, so the regularization gets smaller as a weight comes closer to zero, with the update depending only on $$\epsilon\Delta E(w)$$.

So the main argument for this to my knowledge is that $$\lim_{w\rightarrow0}\lambda = \lambda\\ \lim_{w\rightarrow0}\lambda w = 0$$

• Thanks!! 'more aggressive' -> Clear explanation!! – douner Jan 4 '20 at 7:40

In my case what makes sense is that for the derivative of an absolute value (L1) forces values to be cero, while for L2(quadratic), the slope of the derivative is smoother.