0
$\begingroup$

I'm learning Data Science by enrolling on different courses, and I've recently learnt something that seems very interesting to apply when doing linear or logistic regression models: regularization.

In the screenshot below, the course that I'm currently doing, shows that regularizing these models include an additional factor in the formula of the cost function (which I've circled in red). This factor helps the problem of over-fitting by adding cost when we have many input features.

enter image description here

Now, during this course, we're creating the formulas and functions ourselves to better understand the mechanisms behind these models. However, I would prefer to continue using the functions from sklearn later.

I would like to know if the LinearRegression() and LogisticRegression() models from sklearn, when trained, include (or have any parameter to make them include) regularization.

Thanks everyone!

Improve this question
$\endgroup$
1
  • $\begingroup$ Not those specific functions, however you do have Ridge(), Lasso() and such. $\endgroup$
    – user2974951
    Aug 18, 2022 at 7:51

1 Answer 1

Reset to default
3
$\begingroup$

In sklearn they are presented in a different way as you expected.

Linear regression is without any regularization term, if you look for a regularized version as the one you showed you are looking for a Ridge with alpha = 1.

Logistic regression instead has a parameter called penalty{‘l1’, ‘l2’, ‘elasticnet’, ‘none’}, default=’l2’ that lets you choose the desired penalty.

I suggest you to spend some time on sklearn documentation, it's complete and very well presented!

Improve this answer
$\endgroup$
2
  • $\begingroup$ On a practical level, it's then better to always use Ridge instead of regular LinearRegression(), or are there some considerations that will lead us to the best option? $\endgroup$
    – Álvaro V.
    Aug 18, 2022 at 12:07
  • $\begingroup$ On the practical level, use the regularized one if there is risk of overfitting such as in scenarios with small datasets and lot of variables otherwise go with standard regression since it is easier to train (therefore faster) $\endgroup$
    – DaSim
    Aug 18, 2022 at 13:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.