0
$\begingroup$

Normally we would remove features that have high pairwise correlation with another feature before performing regression. But is this step necessary if I am applying L2 regularized logistic regression (since the regularization algorithm would shrink the "irrelevant" feature coefficients to zero anyway)?

$\endgroup$

1 Answer 1

2
$\begingroup$

Yes the L1 regularization will shrink the irrelevant feature coefficients to zero and hence it doesn't require feature selection. In fact it IS a commonly used feature selection technique. So basically you are performing feature selection!!

$\endgroup$
2
  • 1
    $\begingroup$ Do you perhaps mean $L1$ (LASSO) regression? LASSO can (and often does) shrink coefficients to zero (not quite the same as feature selection...run a regression on just the "surviving" features and compare the coefficient estimates), but ridge regression would not be expected to shrink all the way to zero. $\endgroup$
    – Dave
    Oct 22, 2021 at 18:29
  • $\begingroup$ @Dave My bad! Probably a typo. Corrected. $\endgroup$
    – spectre
    Oct 23, 2021 at 7:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.