0
$\begingroup$

I am playing with features (input data) to improve my model's accuracy.

If I have a raw time-series dataframe, does feature engineering mean extracting properties or characteristics of my raw data and feed it as input? Or will the algorithm learn these from the time-series itself?

In other words, should I create a column that is comprised of the moving average, or will the algorithm pick up on moving average from the raw data?

Is feature engineering just the munging of independent variables? Or is it extracting features that are dependent on other raw data?

EDIT:

Here's another question: If I have a categorical feature, would it be better to have it as a one-hot vector (say, 5 binary inputs), or to have it as one input with range [0,4]?

How does one intuitively know the answer to these questions??

Improve this question
$\endgroup$
6
  • 3
    $\begingroup$ Feature engineering is when you do the leg-work. Feature learning is when the algorithm does it. What the model can pick up depends on the model! Deep neural networks, for example, are famous today for being able to learn hierchical (increasingly abstract, complex) features. $\endgroup$
    – Emre
    Commented Aug 9, 2017 at 5:00
  • $\begingroup$ Ooooh I see.. what would be a good example of 'leg-work'? $\endgroup$
    – Landmaster
    Commented Aug 9, 2017 at 5:01
  • $\begingroup$ Calculating summary statistics, performing standardization, eigendecomposition, or stationarity transformation... $\endgroup$
    – Emre
    Commented Aug 9, 2017 at 5:03
  • $\begingroup$ Ah, so the line between what is learnable by the algorithm and what should be engineered is blurry, I suppose.. If that's the case, one might as well feature engineer all the relevant metrics, just to be safe? $\endgroup$
    – Landmaster
    Commented Aug 9, 2017 at 5:05
  • 1
    $\begingroup$ Don't combine unrelated questions. Categorical variables should be encoded. One-hot is fine. You learn these things from books, papers, classes, mentors, forums, and experience... I suggest picking up a copy of The Elements of Statistical Learning. $\endgroup$
    – Emre
    Commented Aug 9, 2017 at 15:18

1 Answer 1

Reset to default
1
$\begingroup$

Feature engineering refers to creating new information that was not there previously, often by using domain specific knowledge or by creating new features that are transformations of others you already have, such as adding interaction terms or as you state, moving averages. A model generally cannot 'pick up' on information it doesn't have, and that is where finesse and creativity comes into play.

Whether you should one-hot or leave a feature as categorical depends on the modeling approach. Some, like randomForest will do fine with categorical predictors; others prefer recoding.

Intuition on these questions comes with practice and experience. There's no substitute for trying out and comparing toy examples to see how your choices affect outcomes. You should take the time to do that, and intuition will follow.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.