I am aware of below approaches of feature selection
a) Feature Importance methods which are available in tree based models like
Random Forest and
statsmodel.logistic regression which in it's summary output provide us the results which contains whether
variables are significant or not (P-value)
SelectKbest which uses
Chi-square etc to compute the influence of
input variable on
But unfortunately with methods
c, it doesn't consider the feature interaction. Am I right? It works by considering each column to the target variable
Whereas with methods
a it returns the ranking but we aren't sure about whether they are significant or not.
Is there anyway to know from
Feature Importance whether the Features are significant or not? I understand
features occurring in top 4-5 places could be significant but is there anyway to test/validate this?
Or is it like I pick each feature (out of say top 20 assuming they have a role) from feature importance result and do a
SelectKbest test or
How can I know that the features that I select from
Feature importance model are significant?