0
$\begingroup$

Suppose I have a response variable y and and a set of feature variables (x1, x2 ... xn). I wish to find which of x1...xn are the best features for y in a regression problem (the relationship might not be linear).
Is there any way I can do this kind of feature selection without using any correlation measure or regression function in the process (i.e. I cannot use any filter or wrapper methods)?

$\endgroup$
1
  • $\begingroup$ correlation measures linear relationships anyway. $\endgroup$
    – Nikos M.
    May 26, 2021 at 12:14

3 Answers 3

1
$\begingroup$

If you do not want to use filter or wrapper feature selection methods, then you can use tree based algorithms to find the feature importance of all the features. You can use Random Forest, LighGBM, XGBoost, CatBoost for this purpose. CatBoost is an interesting one as it can work with categorical features.

Also you can use L1 or L2 regularization to select the best features. Read up on Ridge and Lasso algorithms.

A word of caution though. Feature selection methods are to be taken with a pinch of sale. Never solely rely on feature selection methods. The best feature selection method IMO is filtering out features based on domain knowledge.

$\endgroup$
0
$\begingroup$

You can train a LightGBM Regressor. LightGBM Regressor feature selection methods embbed in it. You can direct plot them to see which features are important. See this link. LightGBM Plot Importance

$\endgroup$
-2
$\begingroup$

Google up scikit learn feature selection. You should look to use the f_regression and mutual_info_regression to identify the best features for the problem at hand.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.