Multivariate time series forecast with VAR confusion

Question

I am new to time-series forecasting. I am working now on a task in which I have a data set, containing samples of approx. 15 variables for every hour for several years. Then, I have a test data set (continues at the next time step where training data ended) containing values for all the variables except one. My task is to build a model using training data that can predict that one variable in the test data set.

From reading online, I understood I could use vector autoregression (VAR). I have read many tutorials such as this one. I understand most of it except one thing. When it comes to predicting, they (in the tutorials) predict all the variables. However, I would like to do something different: I would like to predict just the one target variable. And of course take into account values of the other variables in the test data set.

To illustrate this, let's say Var Z is my target variable and this is my training set:

       Var X    Var Y     Var Z  
 Day 1     11       20       30
 Day 2     22       40       60
 Day 3     33       60       90

Then this is my test set for which I want to predict Var Z:

       Var X    Var Y     Var Z  
 Day 4     44       80       ??
 Day 5     55       84       ??
 Day 6     66       88       ??

But in the tutorials I have seen so far, they always predict all variables!

Question: How to specify I want to forecast only a single variable for certain timestamps and take into account values of other variables at those timestamps? Is VAR not the right tool to use?

I would be most grateful if someone could point me in the right direction. I use Python.

Answer 1

Since you specifically mention Python, one option is the Prophet package.

The model fitting would be something like:

# Create the pandas DataFrame
import pandas as pd

data = [['2021-01-01', 11, 20, 30],
        ['2021-01-02', 22, 40, 60],
        ['2021-01-03', 33, 60, 90]] 
df = pd.DataFrame(data, columns = ['Day', 'X', 'Y', 'Z'])
df['ds'] = pd.to_datetime(df.Day) # Convert day to datetime for prophet
df['y'] = df.Z                    # Give prophet specific name

# Fit multivariate timeseries 
from fbprophet import Prophet

prophet = Prophet()
prophet.add_regressor('X')
prophet.add_regressor('Y')
prophet.fit(df) 

Answer 2

In essence, you want to both incorporate the past historical values of the target timeseries and the (past and) current historical values of other timeseries to predict the current value of the target timeseries.

In the tutorial you gave, they define the VAR[1] model with two time series ($Y_{1, t}$, $Y_{2, t}$) as:

$$ Y_{1,t} = \alpha_{1} + \beta_{11,1}Y_{1,t-1} + \beta_{12,1}Y_{2,t-1} +\epsilon_{1,t}$$

$$ Y_{2,t} = \alpha_{2} + \beta_{21,1}Y_{1,t-1} + \beta_{22,1}Y_{2,t-1} +\epsilon_{2,t}$$

where the author defined $\alpha$ as the intercept, $\beta$ the lag coefficients and $\epsilon$ as the error term.

Let us consider $Y_{1,t}$ as the target time series. It seems that it could be modified such that it incorporates the past and current values of $Y_{2,t}$ as follows:

$$ Y_{1,t} = \alpha_{1} + \beta_{11,1}Y_{1,t-1} + \beta_{12,1}Y_{2,t-1} + \beta_{12,0}Y_{2, t} + \epsilon_{1,t}$$

Here, the term $\beta_{12,0}Y_{2, t}$ has been added, which signifies the dependence on the current value of the other time series. In fact, this equation is the only relevant one when determining the $\beta$ parameters, as $Y_{2,t}$ is known in the test set.

Generalizing this to an arbitrary number of time series $n$ and lag $l$, the model for your target timeseries becomes:

$$ Y_{1,t} = \text{VAR}[l] + \sum^{n}_{i=2}\beta_{1i,0}Y_{i, t}$$

with VAR$[l]$ indicating the original VAR model from the article.

EDIT:

Worked example based on sample data you provided. We have time series $X,Y,Z$. Let us model our time series with lag = 1. Then our model for $Z_{t}$ becomes:

$$Z_{t} = (\alpha + \beta_{ZZ, 1}Z_{t-1} + \beta_{ZY,1}Y_{t-1} + \beta_{ZX,1}X_{t-1} + \epsilon_{t}) + \beta_{ZY,0}Y_{t} + \beta_{ZX,0}X_{t}$$

The values between brackets is standard vector autoregression. We can then use a simple linear regression procedure. Note that we only have two samples to use for fitting, because at Day 1 there are no previous timepoints. Thus we have the following equations:

$$Z_{2} = 60 = (\alpha + \beta_{ZZ, 1}Z_{1}+ \beta_{ZY,1}Y_{1} + \beta_{ZX,1}X_{1}) + \beta_{ZY,0}Y_{2} + \beta_{ZX,0}X_{2}) = (\alpha + \beta_{ZZ, 1}30 + \beta_{ZY,1}20 + \beta_{ZX,1}11) + \beta_{ZY,0}40 + \beta_{ZX,0}22)$$

$$Z_{3} = 90 = (\alpha + \beta_{ZZ, 1}Z_{2}+ \beta_{ZY,1}Y_{2} + \beta_{ZX,1}X_{2}) + \beta_{ZY,0}Y_{3} + \beta_{ZX,0}X_{3}) = (\alpha + \beta_{ZZ, 1}60 + \beta_{ZY,1}40 + \beta_{ZX,1}22) + \beta_{ZY,0}60 + \beta_{ZX,0}33)$$

Using python we can solve this as follows:

from sklearn.linear_model import LinearRegression

#Z-vals
Z_1 = 30
Z_2 = 60
Z_3 = 90

#Y-vals
Y_1 = 20
Y_2 = 40
Y_3 = 60

#X-vals
X_1 = 11
X_2 = 22
X_3 = 33

LR = LinearRegression()

result = LR.fit([[Z_1, Y_1, X_1, Y_2, X_2],
                [Z_2, Y_2, X_2, Y_3, X_3]],[Z_2, Z_3])

print(f'Alfa is {result.intercept_}')
print(f'Parameters are {result.coef_}')

which yields

Alfa is 21.95159629248201
Parameters are [0.46343975 0.30895984 0.16992791 0.30895984 0.16992791]

And thus our predictive model for Z has become (after rounding):

$$Z_{t} = 21.95 + 0.46Z_{t-1} + 0.31Y_{t-1} + 0.17X_{t-1} + 0.31Y_{t} + 0.17X_{t}$$

Answer 3

You should read this post.

It's a long one but it's a useful one. It relies on LSTM for time series forecasting and does a very good job of illustrating how framing the problem, ie: how to you define what's the training and the target data, influence what the model is going to predict.

In your case, it's gonna show that you only need using you n-1 Var present in both data set and use the remaining one in the train data as the target. How you decide frame it is up to you, whether you want to do a step by step prediction or predict a vector of values.