0
$\begingroup$

I would greatly appreciate if you could let me know whether I should omit highly correlated features before using Lasso logistic regression (L1) to do feature selection.

In fact, I want to use logistic regression with L1 to do prediction as well as feature selection. However, some of my features are highly correlated e.g., -1 or 0.9. Should I omit them before applying Lasso or let the Lasso decide it?

Really, I read in Mr. Raschka’s book (Python Machine Learning) that

regularization is a very useful method to handle collinearity (high correlation among features).

However, this kernel (by referring to Wikipedia) states that keeping correlated features in the model would adversely affect the feature selection but it doesn't impair the predictions.

$\endgroup$
1
$\begingroup$

Use scikit-learn package. In your case you need find sklearn.linear_model.LogisticRegression

and User guide

It's clear enough for understanding. You needn't special actions to win collinearity. But instead of linear method you can use non parametric algorithm like random forest sklearn.ensemble.RandomForestClassifier.

Сompare the results of the logistic regression & random forest on the test data

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.