0
$\begingroup$

In a linear model, regularization decreases the slope. Do we just assume that fitting a lin model on training data overfits by almost always creating a slope which is higher than it would be with infinite observations instead? What is the intuition?

$\endgroup$
0
$\begingroup$

Regularization is used to help smooth multi-dimensional models. Take this example,

y = x_1 + eps*(x_2 + ... + x_100)

Let's say eps is very small. It doesnt seem very useful to store those 99 coefficients, isn't it? How do we manage to fit a model in such a way that we drop negligible coefficients? This is exactly what L1-regularisation does!

Each other type of regularisation has another geometric intuition.

$\endgroup$
1
  • $\begingroup$ Thank you for the reply! So it seems like there is no fundamental use in applying regularization to a simple linear regression model with only one variable(Except of-course if you have an optimization problem like in the real world). Your explanation makes lots of sense to me. $\endgroup$
    – AUser240
    Nov 20 '20 at 23:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.