In a linear model, regularization decreases the slope. Do we just assume that fitting a lin model on training data overfits by almost always creating a slope which is higher than it would be with infinite observations instead? What is the intuition?
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
Regularization is used to help smooth multi-dimensional models. Take this example,
y = x_1 + eps*(x_2 + ... + x_100)
Let's say eps is very small. It doesnt seem very useful to store those 99 coefficients, isn't it? How do we manage to fit a model in such a way that we drop negligible coefficients? This is exactly what L1-regularisation does!
Each other type of regularisation has another geometric intuition.
-
$\begingroup$ Thank you for the reply! So it seems like there is no fundamental use in applying regularization to a simple linear regression model with only one variable(Except of-course if you have an optimization problem like in the real world). Your explanation makes lots of sense to me. $\endgroup$ – AUser240 Nov 20 '20 at 23:16
