0
$\begingroup$

I am currently implementing a forecasting model. The whole builds on a regression. The first model is an LSTM network based on Keras and the second a Random-Forest model based on sklearn.

I use the same input values and target variables for both models. The forecasts of the LSTM network are closer to the actual observed values (better rmse and r2) (probably due to the memory / lookback), but the Random Forest model provides better altitudes and depths. It seems that the Random-Forest model has found better dependencies than the LSTM model.

LSTM:

enter image description here

Random-Forest: enter image description here

What is the best way for the LSTM model to understand finer dependencies? Should I use an ensemble or stack method from LSTM + RF? Does it make sense?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Is this the training data? Make sure that you are testing your models on an independent sample. A common way is to split the dataset in two, train on one (say time between 0 and 500) and test on the other. Then look at the R2. The reason I'm asking is that I am very surprised to see such good results. $\endgroup$
    – Gino_JrDataScientist
    Commented May 21, 2018 at 8:39
  • 1
    $\begingroup$ Also: your data might have both a daily dependence e.g. higher call volume in the afternoon) and a weekly dependence (more calls on Monday). Instead of training a model on the whole series of call volume, you might be better off feeding your model one day at a time, that is you cut the time series between 8 am and 5 pm every day. After all, you do not want your model to spend parameters on learning that no calls are made at night or during the weekend. Plus you can add the day of the week as a parameter in the model $\endgroup$
    – Gino_JrDataScientist
    Commented May 21, 2018 at 8:42
  • $\begingroup$ @Gino_JrDataScientist 1.Of course I test on a validation set. 2.The results are only that good because I've done pretty good feature engineering. My concern here is that I bring the advantages of the two models in one. The question is: how? $\endgroup$
    – Meiiso
    Commented May 21, 2018 at 8:58

1 Answer 1

Reset to default
1
$\begingroup$

Some (possibly stupid) ideas for combining the two models:

  1. Make an average of the two forecasts

  2. Instead of a naive average you use LSTM for estimating the coarse daily trend and the RF for the fine details, maybe like so:

    a. Smooth (maybe with a loess or a kernel) the LSTM to get the daily trend

    b. For each day, subtract the mean volume call from the prediction of the RF, to get the fine details

    c. Add the forecast of two models

General suggestions for improving your model:

  1. Instead of training a model on the whole series of call volume, you might be better off feeding your model one day at a time, e.g. you cut the time series between 8 am and 5 pm every day. After all, you do not want your model to spend parameters on learning that no calls are made at night or during the weekend.

  2. Add the day of the week as a parameter in the model, as well as some information on the call volume in the past (like previous day) if you think that there might be some time correlation between consecutive days.

  3. If you think that the day of the week has a strong effect on the call volume (e.g. if each day has a special, recurring pattern) then you might consider creating a separate model for each day of the week.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Hi Gino: 2. Of course I have the weekday as a feature 3. I think it would be too much if I train a prognostic model for every day of the week. As explained above, this question is more about how I can combine the two advantages of the models into one. The LSTM net delivers pretty good results in quantity, the RandomForest gives better results in detail. $\endgroup$
    – Meiiso
    Commented May 21, 2018 at 9:03
  • $\begingroup$ Ok. What about 1? $\endgroup$
    – Gino_JrDataScientist
    Commented May 21, 2018 at 9:07
  • $\begingroup$ I do not think that will bring the desired results. $\endgroup$
    – Meiiso
    Commented May 21, 2018 at 9:14
  • $\begingroup$ Ok. I modified my answer with some stupid ideas for combining the two models. Hope it helps! $\endgroup$
    – Gino_JrDataScientist
    Commented May 21, 2018 at 9:16
  • 1
    $\begingroup$ I still think that using parameters for learning that no calls are done on weekends and nights is a waste :) $\endgroup$
    – Gino_JrDataScientist
    Commented May 21, 2018 at 9:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.