What are the top (including new, if any) algorithms to perform features selections without removing or altering the missing data points ? Thanks


1 Answer 1

  1. Random Forest: Random Forests are often used for feature selection in a data science workflow. This is because the tree-based strategies that random forests use, rank the features based on how well they improve the purity of the node.

  2. Recursive Feature Elimination (RFE): RFE is a popular feature selection algorithm that recursively removes attributes and builds a model on those attributes that remain. It uses model accuracy to identify which attributes (and combinations of attributes) contribute the most to predict the target attribute.

  3. Lasso Regression: Lasso Regression is a popular method to reduce overfitting in a model. It does this by making the coefficients of the features that do not contribute to the model output zero, effectively eliminating them.

  4. Ridge Regression: Ridge Regression is a method that adds a degree of bias to the regression estimates, which reduces the standard errors. It might not eliminate the features like Lasso, but it will assign them lower weights.

  5. Elastic Net: Elastic Net is a middle ground between Ridge Regression and Lasso. It penalizes the absolute and square size of the regression coefficients.

  6. Boruta: Boruta is an all-relevant feature selection method. It tries to capture all the important, interesting features you might have in your dataset with respect to an outcome variable.

  7. XGBoost: XGBoost provides a built-in feature importance plot which ranks the features based on their importance in the model.

  8. LightGBM: LightGBM also provides a built-in feature importance plot which ranks the features based on their importance in the model.

  9. Variance Threshold: This method removes features whose variance doesn’t meet a certain threshold. It assumes that features with a higher variance may contain more useful information.

  10. SelectKBest: This method selects features according to the k highest scores of a given scoring function.

Remember, these algorithms do not remove or alter missing data points, but they may be affected by them, so it's always a good idea to handle missing data appropriately before feature selection. Caution, if you want to make a model without touching NaN values and you want a model that can handle it directly then you can go with XGBoost, CatBoost

  • $\begingroup$ This is gold! Thanks a lot. $\endgroup$
    – Zak
    Jul 12, 2023 at 5:36
  • $\begingroup$ Happy to help you. You can close the question if you are satisfied $\endgroup$ Jul 12, 2023 at 5:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.