# Feature Selection with one-hot-encoded categorical data

I have a dataset with 400+ columns. Almost 90% of these are categorical data with One-Hot-Encoding (OHE). I'm using the dataset for a classification problem.

My professors asked me to perform feature selection using sequential forward selection (mlxtend).

Is there really a point of doing this since it is also very time consuming? Is it logical to remove categorical data? If so, what would be the k_features number for sfs that I should use? Or is the method(sfs) even suited for this?

• what method do you use? What is sfs? Just removing categorical features is not a good idea, since they can be important. Feature selection is very much dependent on the method. If you use logit for instance, you can simply (and extremely efficient) use Lasso. However, features selected by Lasso will not necessarily also be relevant in (e.g.) boosting. – Peter Jun 1 '19 at 21:22
• He told me to use sequential forward selection(sfs) from mlxtend library. Is feature selection usable for categorical data ? Is Lasso better for these features ? – dungeon Jun 1 '19 at 21:45
• Docs say mlxtens is „A library of Python tools and extensions for data science“. So it is still not clear what type of classifier/regressor you use. However, if the problem was phrased exactly like you stated in the question, I guess you need to go all the way and do the daunting task. – Peter Jun 1 '19 at 21:49
• I'm using support vector classification if that's what you mean. I will run this problem with 4 classifiers KNN, SVC, Random Forest and Logistic Regression should I do feature selection for every classifier differently? – dungeon Jun 1 '19 at 21:55
• in tendence: yes. For Logistic Regression (as noted above) Lasso is THE way of selecting features. For the remaining ones, its not so clear. scikit-learn.org/stable/modules/generated/… – Peter Jun 1 '19 at 22:12