Normally we would remove features that have high pairwise correlation with another feature before performing regression. But is this step necessary if I am applying L2 regularized logistic regression (since the regularization algorithm would shrink the "irrelevant" feature coefficients to zero anyway)?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Yes the L1 regularization will shrink the irrelevant feature coefficients to zero and hence it doesn't require feature selection. In fact it IS a commonly used feature selection technique. So basically you are performing feature selection!!
-
1$\begingroup$ Do you perhaps mean $L1$ (LASSO) regression? LASSO can (and often does) shrink coefficients to zero (not quite the same as feature selection...run a regression on just the "surviving" features and compare the coefficient estimates), but ridge regression would not be expected to shrink all the way to zero. $\endgroup$– DaveCommented Oct 22, 2021 at 18:29
-
$\begingroup$ @Dave My bad! Probably a typo. Corrected. $\endgroup$– spectreCommented Oct 23, 2021 at 7:32