4
$\begingroup$

We have a data set with a set of N engineered features in each row. These features are actually a time series.

I am being asked to use them to train a random-forest-type model for classification. However no effort is being made to treat the features as a time series such as account for correlations or seasonality. The model formula is just identical to that of an additive linear model: y ~ b0 + b1 + b2...b120

The lead insists that the random forest, because it learns complex interactions, will learn the appropriate correlations foe the time series. Is this right?

$\endgroup$

2 Answers 2

3
$\begingroup$

Random Forest treats each row independently, so it will ignore any kind of time series correlations. You can verify this by shuffling your data before training, your model (aside from non-stochastic nature of Random Forest) will be the same.

$\endgroup$
3
  • $\begingroup$ sorry if I was unclear but what about correlations within the row. the rows themselves are each a time series of information $\endgroup$
    – barnhillec
    Nov 12, 2021 at 20:29
  • $\begingroup$ @barnhillec, Just to clarify, your features are something like: day 1, day 2, day 3, ... day n, day n+1? If so, Random Forest will not care about order of features either. $\endgroup$
    – Akavall
    Nov 12, 2021 at 20:53
  • $\begingroup$ right. no it wouldn't care about the order of the input. but would it learn the correlations that were present in the series the way a time series model would? $\endgroup$
    – barnhillec
    Nov 12, 2021 at 20:54
2
$\begingroup$

XGBoost, LightGBM, and CatBoost are almost always better than Random Forest in accuracy and they are similar to Random Forest. LightGBM is also much faster to calculate. They all work best if the time series features are added to the data. Lags, moving averages, standard deviations of different number of periods help improve accuracy significantly.

Just look at M5 Forecasting Accuracy Wallmart Kaggle competition (https://www.kaggle.com/c/m5-forecasting-accuracy). The first 5 places are 4 LightGBM models and 1 DeepAR model.

The first place and, if I remember correctly, all other first places with LightGBM models, had:

  1. lags
  2. moving averages
  3. moving standard deviations
  4. features based on price

About 10 years ago the winning models were Random Forest for classification and different time series algorithms like Exponential Smoothing for time series. But now not any one of the winning solution uses Random Forest. And the boosting algorithms gain significantly from feature engineering.

In my opinion this kind of feature engineering was a new development several years ago that allowed the tree algorithms to beat simple and even more complicated usual time series algorithms when you have additional external data (like prices). This should also apply to Random Forest.

I am working full-time as a data scientist at a consulting company making demand forecasting to improve supply chain processes. I was involved with about 6 clients to create demand forecasts and I was once competing with a competitor for the best accuracy and won.

$\endgroup$
2
  • $\begingroup$ Some of this is super helpful, adding some lags or averages would be one way to help the time series along. I think my question is more technical though. Is the random forest going to learn the time series correlations? $\endgroup$
    – barnhillec
    Nov 13, 2021 at 1:14
  • $\begingroup$ If you only have one last data point as your "X" then no. y_pred is a function of "X" and will only deliver the result based on the one value provided. But if you have some of the previous data in "X"(I call it lags) then the model can learn some correlations. Though it is not as efficient as also adding other features. $\endgroup$
    – keiv.fly
    Nov 13, 2021 at 15:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.