Feature selection with "overly important" features

Question

I am very new to machine learning modeling, but I encountered a feature selection problem that I hope can get your insights on:

• For example, I have A,B,C,D as my independent variables and y as my dependent variable. The end user is more interested in C & D's impact on y since A and B are factors that the user don't have much power to change.
• But in the modeling, we see that A and B have very large feature importance in predicting y, while C and D have low prediction power.
• In this case, should I train the model only based on C&D or I should train the model based on ABCD？
• or is there any feature engineering I should do?

Answer 1

I think it's a matter of clearly defining the target task:

• If the goal is to predict $y$ as accurately as possible and all 4 independent variables are available, then in general there's no reason not to use the 3 variables.
• If the goal is to predict $y$ using C and D only, or to calculate the impact of C and D on their own on $y$, then only C and D should be used, of course.

It might also be some kind of optimization problem where the goal is for instance to maximize $y$ using C and D, this would be a different problem.