3

There seems to be two possible approaches to your problem : If they are just identification features that you know aren't informative, you should remove them yourself. SelectKBest - like almost any other EDA tools - works on all the features you provide it, there is no way it knows what features are supposedly uninformative identification features and which ...


2

Its best to remove such a variable. Reasons are following: Artificial imputation can add bias and result cannot be justified because 99% data for the particular variable was artificially created. The variables/features that you choose for building the predictive model should have low correlation with the target/outcome variable/feature. Because, variable ...


1

Every time you compress the feature space you are losing some information. The original feature engineering stage you outlined sounds like a meaningful compression & might make sense in the context of your problem. The second compression on the other hand might only serve to lose some information. I would only perform the second compression if the ...


1

Exogenous simply means a value that is determined outside the context of your model & is then imposed on your model. Endogenous means the model determines the value. I don't know about "external" as this word seems to depend on context. But you would be right to say these variables are exogenous.


1

There's a good chance that it's a sign of overfitting: the fact that the importance of the features is not stable can be considered as an indication that the model itself is not stable, and this typically happens when it doesn't have enough information in the data to be sure how to use the features. As a result minor variations in the features or data cause ...


1

No, RFE cannot guarantee that it finds the feature subset with optimal score. As with most greedy processes, the point of RFE is to reduce the computational cost (fitting a model for each of the $2^m$ feature subsets), at the cost of perhaps not finding the actual optimum (but hopefully "close enough"). See also https://stats.stackexchange.com/...


1

This important point is missing: SFS is suitable as it has no assumption for features to be categorical or numerical. However, one-hot encoding is redundant when you are planning to use SFS. You just make the process longer by one-hot encoding since by doing so SFS needs to check more number of features than what it actually is.


Only top voted, non community-wiki answers of a minimum length are eligible