# Ridge and Lasso Regularization

Recently, I started working on Ridge and Lasso regularization for Linear and Logistic Regression. My doubts are given below:

1. Is the penalty the same (by same proportion) for all the coefficients or is it based on variable importance? If it is the latter I believe we can directly apply regularization rather than spending time in feature selection.
2. Whether the Multi-collinearity is taken care by ridge and lasso regularization?

Thank you.

• 1. The penalty is the $L_p$ norm, so large features are punished. The response variable is not taken into account. 2. LASSO is sensitive to it, so it is common to combine it with ridge regression to yield "elastic net" regression (L1+L2). This selects the entire collinear group. If you don't want that, drop the redundant ones first. – Emre May 8 '18 at 4:54
• I think Emre answers the question. – HelloWorld May 8 '18 at 5:02

The penalty of both Lasso and Ridge is proportional to the magnitude of the weight. That is, the penalization added to the cost function is $\lambda ||\omega||_2$ or $\lambda ||\omega||_1$.