2
$\begingroup$

I set up a forecasting model that predicts call data. The forecast model uses a random forest regression model.

Data: I have call data about every day in 15 minutes intervals of a year since 2013.

Here is a plot of the accumulated values over months:

Calldata

It can be clearly seen that call data has almost doubled in 2017 over 2016. This trend should also be observable for the next few years.

features:

First, the format of my data:

 DATE                     CALL
 ....
 2017-10-23 10:15:00.000    259
 2017-10-23 10:30:00.000    292
 2017-10-23 10:45:00.000    309
 ....

From this I extracted the following features: I have extracted the following features to predict my target variable Y (call data):

-Weekday
-Month
-Holiday (yes / no)
-Interval of the day

So I ask my model:

What is the call volume of a day and interval with the following features?

I have used the years 2015-2016-2017 to train the model. However, the model does not give the desired prognosis.

He even predicts the days for 2017 wrong. Although I gave him the data as training data.

Questions:

- Should I work on my features?

- How do I show my forecasting model that the data will double year by year 
  as observable since 2016?
$\endgroup$
2
$\begingroup$

Before going into modelling, I guess you can do a bit more of exploratory analysis(month by month, year by year). If you find any trend or seasonality and so on.

Why did you go to RF directly without using basing Techniques like ARIMA, ARMA, Exponential Smoothening and AR so on.

Sometime RF might not give you as good results as Base models, I think you don't have trend, this is from your graph(but not certain). If you can try doing some research and see if there are some external factors which are effecting your demand. Why did that happen and what is the root case for it.

To your model to understand, it needs some feature which explains its spikes somehow, it can achieved by doing feature engg/ research

$\endgroup$
7
  • $\begingroup$ Hi @Toros91, thanks for your help. So you say that from my data set, the information I need can not be pulled out and I have to use other sources?So you recommend using other sources aswell?I need a stable forecasting model. that's why I switched to RF. For call data which are steady that worked wonderfully. But for call data that rises as above year by year makes my forecast model problems. $\endgroup$ – Meiiso Nov 16 '17 at 9:47
  • $\begingroup$ yeah try using ARIMA, ARMA models too, yes if you can do some research most likely that you would end with some good external factor which explains you about those spikes, as I don't know about your business, can explain you with an example: if there is sale of pens in the month of March there is some spike the reason for that is school reopen. So, students buy pens during that period of time. This is just an example. $\endgroup$ – Toros91 Nov 16 '17 at 9:51
  • $\begingroup$ Would you recommend a specific machine learning model for my case? Yes you are right, I will continue to research and analyze the reasons for the increase $\endgroup$ – Meiiso Nov 16 '17 at 10:06
  • $\begingroup$ Here you need to use time series models not regression model. RF does forecasting but not every other machine learning algorithm does that. $\endgroup$ – Toros91 Nov 16 '17 at 10:09
  • $\begingroup$ i tried Times series models but that didnt work well. I want to predict minimum the next +3 Month. With Regression i came to the best results. Or what kind of time series model would you recommend? I had a moving window without features. $\endgroup$ – Meiiso Nov 16 '17 at 10:11
2
$\begingroup$

Random forest, and tree-based models in general, do not handle trends well.

The reason is simple: inside any decision tree, there are discrete rules such as: $$ y = \begin{cases} y_1, & \text{if } x > c \\ y_2, & \text{if } x <= c \end{cases} $$ This is a tree of depth 1 (so-called "stem"), but deeper trees obey the same logic. The variable $x$ and constants, $y_1$, $y_2$, $c$ are fit to the train data. And this is the problem: if in the training data $y$ was never higher than $y_1$, your tree will never predict $y>y_1$, even if $y$ is clearly increasing.

On the other hand, linear models (such as XARIMA and its special cases) catch trends very well. But they are poor with non-linearities and feature interplay in your data. In my own experience, the following stacking approach works best:

  1. Fit a simple time-based linear model to your data.
  2. Fit a tree-based model (random forest or boosting) to the residuals of your linear model.

If the linear model is specified correctly, it will catch and remove the non-stationarities in the data. Thus, the tree-based model will be predicting stationary residuals and find finer dependencies that the linear model.

This Python example illustrates the issue:

import numpy as np
from sklearn.datasets import make_friedman2
from sklearn.ensemble import RandomForestRegressor
from sklearn.linear_model import HuberRegressor
from sklearn.metrics import r2_score
from sklearn.model_selection import cross_val_predict

# make a difficult dataset with a linear trend
X, y = make_friedman2(n_samples=1000, random_state=1, noise=10)
time = np.arange(1000)
y += time * 1.5
X = np.hstack([X, time[:, np.newaxis]])
X_train, y_train = X[:700], y[:700]
X_test, y_test = X[700:], y[700:]

# build a pure Random Forest
rf = RandomForestRegressor(random_state=1, n_jobs=-1).fit(X_train, y_train)
y_rf = rf.predict(X_test)

# build a pure linear model
linear = HuberRegressor().fit(X_train, y_train)
y_lin = linear.predict(X_test)

# build a stack of two models
lin_resid = y_train - cross_val_predict(linear, X_train, y_train)
rf.fit(X_train, lin_resid)
y_stack = y_lin + rf.predict(X_test)

print(r2_score(y_test, y_rf))    # R2 on test data is only 0.34
print(r2_score(y_test, y_lin))   # R2 due to time trend is 0.86 
print(r2_score(y_test, y_stack)) # R2 of combined model is 0.95
$\endgroup$
8
  • $\begingroup$ Thanks for your help. I have some other Questions, is there a way to come in contact with you? $\endgroup$ – Meiiso Nov 16 '17 at 22:15
  • $\begingroup$ @Meiiso yes, you can write me on Facebook www.facebook.com/dale.david.fluteman $\endgroup$ – David Dale Nov 17 '17 at 2:16
  • $\begingroup$ I'll write you in the next days :) One Question for now: i tried your stack of two models for my case and the r2 score function does not work for my case. the output is "Found input variables with inconsistent numbers of samples: [13112, 171924544]". Why y_stack has in my case more values as the y_test? i took your example...for the other r2 score calculation (linear model and rf) that works $\endgroup$ – Meiiso Nov 20 '17 at 11:01
  • $\begingroup$ @Meiiso please give me the code that produces this error. I don't have extrasensory powers to debug code without seeing it. $\endgroup$ – David Dale Nov 20 '17 at 11:09
  • $\begingroup$ Okay, i'll write you on Facebook with my jupyternotebook + Data $\endgroup$ – Meiiso Nov 20 '17 at 11:24
1
$\begingroup$

Considering your data (The sample you show above) I would suggest tbats() function from forecast package in R. Because your data might have hourly as well as daily seasonality which suggests us to use "TBATS model (Exponential smoothing state space model with Box-Cox transformation, ARMA errors, Trend and Seasonal components)"

References: De Livera, A.M., Hyndman, R.J., & Snyder, R. D. (2011), Forecasting time series with complex seasonal patterns using exponential smoothing, Journal of the American Statistical Association, 106(496)

Or you could use Dynamic harmonic regression. Reference and examples https://otexts.org/fpp2/complexseasonality.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.