I am working on financial prediction problem(time-series prediction problem).

I think feature engineering is importance in this problem. So i am careful to check the feature's effectiveness. And i perfer linear regression, because i think it's easy to explain, and I am not good at machine learning models

But i think i have another method: build a lot of features without careful check, select good model to handle it.

So, my question is:

In which model, add what kind of features, will harm this model's ability?

And Is there any models, i can just add without worrying the harm of trash features? in this model, what should i pay attention to?


In very simple words,
Those models which use all the features of the instances will suffer from irrelevant features e.g. Neural Nets, KNN etc.
While the models which have an internal strategy to compares the features to decide the best while training will not suffer(at least for this reason) e.g. Tree

Here is a snap from "The Elements of Statistical Learning, by Trevor Hastie, Robert Tibshirani, Jerome Friedman

enter image description here

You can also read this section 10.3 Effect of Irrelevant Features from "Feature Engineering and Selection: A Practical Approach for Predictive Models by Max Kuhn and Kjell Johnson"

You can try these approaches to filters the features -

  • Using correlation with the target
  • Using Feature Importance
  • Using L1 penalty

Also, keep in mind that you might face other issues because of this and the models may suffer indirectly e.g.

  • High dimensionality
  • Issue of Trees "default feature importance approach" with high Cardinal features etc.
  • 1
    $\begingroup$ great, thanks. very professional $\endgroup$ – nick Apr 4 at 14:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.