I have a time-series (sensor data) with 2 variables x and y.

                         x     y
2020-05-20 05:15:00    1192  355675
2020-05-20 05:30:00    1171  357600
2020-05-20 05:45:00    1260  392680

Rather than traditional time-series prediction with autoregression, my goal is to estimate y given x and the timestamp.

I've gotten acceptable results using non-temporal models (regression, ensembles, etc), but this way I'm throwing away the time information. The other model I've considered using is Vector Autoregression - given that Granger Causality exists between the two variables - but my understanding is that this requires autoregression input from both variables.

What would be the best choice of model for the relationship between these variables over time? My goal is to be able to use only sensor x, and estimate probably values of y.


You could start using a dynamic regression model, they are used when besides the times series to be forecasted, you have another series where you could use it.

Essentially, this is a usual regression model, also called Autoregressive Distributed Lags (ADL), where you include not only the dependent variable $ y_{t} $ but also explanatory variable $ x_{t} $. Following equation exemplifies that:

$$ y_{t} = c + a_{1} y_{t-1} + b_{0} x_{t} + b_{1} x_{t-1} + \epsilon_{t} $$

Now notice that equation above uses $ x_{t} $, meaning that at time you have variable $ x $ so you could use it to forecast $ y_{t} $. This is not always possible, so take care of data leakage.

You can read more about it in Forecasting: Principles and Practice - Chapter 10 Dynamic regression models

Having said all that, you can also improve your ML approach by computing temporal features. Rolling Averages up to lag t, Rolling sum, max, min, and all lags variables could be example of feature engineering you could apply and still use a machine learning approach. Also, if you use python, there is a fresh package called sktime where you can use Reduced Regression.

Example from their notebook (link above):

from sktime.forecasting.compose import ReducedRegressionForecaster

forecaster = ReducedRegressionForecaster(regressor=regressor, window_length=12, strategy="recursive")

y_pred = forecaster.predict(fh)
plot_ys(y_train, y_test, y_pred, labels=["y_train", "y_test", "y_pred"]);
smape_loss(y_test, y_pred)
  • $\begingroup$ Thanks, this is helpful. But IIUC it still involves using y to predict y, right? $\endgroup$ – Josh Friedlander Aug 13 '20 at 12:25
  • 1
    $\begingroup$ Yes, in the second case still we use y, but in a regression fashion. I am not an expert in VAR, but I think you could use them. But if your task is forecasting reliably, I would go for a feature engineering approach, calculating statistics of past values, lags values, etc and use a ML robust model. Just watch out for leakage and not calculating features using any information for $ y_{t} $. $\endgroup$ – Victor Oliveira Aug 13 '20 at 12:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.