Machine Learning based Multivariate Time Series Prediction - How to create supervised data format

Question

Q1:

I have a multivariate time series dataset. For each timestep, there are 11 features and 1 output. I am going to use supervised ML to predect the output. I understand that in univariate cases, if I am going to use the past 3 days to predict the t-th day, the dataset will be formatted as

x(t-3) | x(t-2) | x(t-1) | x(t)

, where x(t) is the output to predict. 

How should I format the dataset when it is a multivariate problem?

I saw that in some kernels, the problem is formatted as

x12(t-3) | x12(t-2) | x12(t-1) | x1(t), x2(t), ..., x12(t)

, where x12(t) is the output to predict. 

In this case, variables x1 to x11 for the past 3 days are ignored.

However, these variables may be important in my case. Can I format the problem into

x1(t-3),...,x12(t-3) | x1(t-2), ..., x12(t-2) | x1(t-1),..., x12(t-1) | x1(t), x2(t), ..., x12(t) 

?

(some of the features are just day, month, day of week, etc. created from the datetime index)

Q2:

With only 11 features, is it necessary to conduct feature selection?

Answer 1

Regarding how to handle multivariate time series problem, I believe that GitHub link Timeseries multivariate will be helpful.

You have to change n_inputs and n_outputs as 12 and 1 respectively.

Answer 2

I'll write a more generalized answer, and hopefully, you'll find it helpful.

We start of with a dataset $X$ with $N$ samples, where each sample $F_x$ features, name $x_i \in \mathbb{R}^F$ and $i\in[0,N-1]$.Note that $X$'s shape is $(N,F_x)$. We also have $N$ targets, each could also be multivariate. we call these targets $Y$, with shape $(N,F_y)$. (usually, $F_y=1$)

To properly sample features and targets from $X,Y$, one must first define a history sample size and a prediction size. let $H$ denote the history size, and $P$ the prediction size. a features,targets tuple is a tuple $(x,y)$ such that the shape of $x$ is $(B,H,F_x)$ and the shape of $y$ is $(B,P,F_y)$.

We can think of $x$ as a randomly sampled batch of $H$ inputs, each of size $F_x$, and $y$ as a randomly sampled batch of $P$ targets, each of size $F_y$

               

Using PyTorch, one could implement a custom dataset with the following (semi-pseudo-code) __getitem__ method:

def __getitem__(self, idx: int) -> Tuple[torch.Tensor, torch.Tensor]:
    n_windows = N - H - P + 1
    win_idx = idx // n_windows
    features_window = features[win_idx: win_idx + H]
    target_window = targets[win_idx + H: win_idx + H + p]
    return features_window, target_window

As per your second question, the small number of features alone does not answer the question of whether feature selection should be used. A more suitable question would be the relation between those features, and if they correlate such that redundancy is present. Of course, one could always try and see the results with and without some of the features.