I am very new to machine learning modeling, but I encountered a feature selection problem that I hope can get your insights on:

  • For example, I have A,B,C,D as my independent variables and y as my dependent variable. The end user is more interested in C & D's impact on y since A and B are factors that the user don't have much power to change.
  • But in the modeling, we see that A and B have very large feature importance in predicting y, while C and D have low prediction power.
  • In this case, should I train the model only based on C&D or I should train the model based on ABCD?
  • or is there any feature engineering I should do?

1 Answer 1


I think it's a matter of clearly defining the target task:

  • If the goal is to predict $y$ as accurately as possible and all 4 independent variables are available, then in general there's no reason not to use the 3 variables.
  • If the goal is to predict $y$ using C and D only, or to calculate the impact of C and D on their own on $y$, then only C and D should be used, of course.

It might also be some kind of optimization problem where the goal is for instance to maximize $y$ using C and D, this would be a different problem.

  • 1
    $\begingroup$ I also suggest to do a variable study to improve you're knowledge about them. For example you can do some analysis like density plot, PCA or correlation. Maybe you discover new information about your data. $\endgroup$
    – Inuraghe
    Mar 17, 2022 at 7:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.