3
$\begingroup$

I am working on a binary classification with 977 rows. class proportion is 77:23. I have lot of high cardinality categorical variables and couple of numeric variables such as Age and quantity.

I would like to know what are some of the automated feature selection packages available in python. Basically, as a data scientist, knowing this list of packages would help me in doing my tasks efficientlty.

I came across the below

a) Featurewiz here

b) sklearn.feature_selection

c) Xverse here

d) SHAP-hypertune here

I am sure there are lot more packages that can be added to this list and am not aware of it.

Can I please have your help to list the other automated feature selection packages in python?

Improve this question
$\endgroup$
10
  • $\begingroup$ Are you using a random forest approach or a deep learning/neural network approach? Your tags do not align with the text in the question, and that makes it tricky to determine what you need. $\endgroup$
    – Dave
    Feb 21, 2022 at 4:48
  • $\begingroup$ I am exploring multiple models. One is Random Forest and other is MLP. I am also trying boosting etc. Since feature selection is common to all ML techniques (and also am using neural network), thought I will use that tag. $\endgroup$
    – The Great
    Feb 21, 2022 at 4:49
  • 1
    $\begingroup$ Featuretools featuretools.alteryx.com/en/stable $\endgroup$
    – Peter
    Feb 21, 2022 at 5:51
  • $\begingroup$ There's also backwards and forwards selection algorithms. Most people caution against their use (particularly if you're running a hypothesis test each iteration) - but it's a quick and dirty approach that may suit your needs. $\endgroup$
    – Cameron Chandler
    Feb 21, 2022 at 7:45
  • 1
    $\begingroup$ github.com/scikit-learn-contrib/boruta_py $\endgroup$
    – Carlos Mougan
    Feb 21, 2022 at 14:11

1 Answer 1

Reset to default
2
$\begingroup$

In addition to these algo ML algorithms with high regularization can do a intrinsic feature selection. This is known as Kitchen Sink Approach. In this all features are pushed to ML model and ML model decides what it is important for it.

For example: L1 regularization in regression can do feature selection intrinsically

Improve this answer
$\endgroup$
4
  • $\begingroup$ thanks for the help. upvoted. Is there any other python package based approaches that you know of? $\endgroup$
    – The Great
    Feb 21, 2022 at 7:13
  • $\begingroup$ Will I be able to apply L1 or L2 regularization with random forest or neural network? Or I have to first build a seperate model to understand the important features (through lasso/ridge) and then later use those features in complex models. Is that what you mean? $\endgroup$
    – The Great
    Feb 21, 2022 at 7:58
  • $\begingroup$ I dispute the notion that L1 regularization is feature selection. Those features are in the model, just with coefficients that are estimated to be zero. If you run an unregularized linear model on the features with nonzero coefficients, you will wind up with different coefficients, meaning that the zero-coefficient features do make it into the regularized, even if only implicitly. $\endgroup$
    – Dave
    Feb 21, 2022 at 16:57
  • $\begingroup$ @TheGreat for neural networks, you can conceptually use $\ell_1$ regularization, but standard stochastic gradient approaches will not be able to get any weights exactly to zero (and actually, it may be quite slow getting close). With random forests, typically are other feature selection methods used rather than l1 regularization; simpy googling "feature selection random forest" will yield some popular approaches. $\endgroup$
    – John Madden
    Feb 21, 2022 at 18:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.