Problems to understand how to create the input data for time series forecasting with a recurrent neural network in Keras

I just started to use recurrent neural networks (RNN) with Keras for time-series forecasting and I found this tutorial Forecasting with RNN. I have difficulties understanding how to build the training data both regarding the syntax and the format of the input data.

Here is the code:

import pandas as pd
import numpy as np
import tensorflow as tf

from tensorflow import keras
from matplotlib import pyplot as plt

# Read the data for the parameters from a csv file

#Delete the first column as it is not used in the tutorial for forecasting
del df['datetime']

data = df.values

n_steps = 168

series_reshaped =  np.array([data[i:i + (n_steps+24)].copy() for i in range(len(data) - (n_steps+24))])

X_train = series_reshaped[:43800, :n_steps]
X_valid = series_reshaped[43800:52560, :n_steps]
X_test = series_reshaped[52560:, :n_steps]
Y = np.empty((61134, n_steps, 24))
for step_ahead in range(1, 24 + 1):

Y_train = Y[:43800]
Y_valid = Y[43800:52560]
Y_test = Y[52560:]

np.random.seed(42)
tf.random.set_seed(42)

model6 = keras.models.Sequential([
keras.layers.SimpleRNN(20, return_sequences=True, input_shape=[None, 6]),
keras.layers.SimpleRNN(20, return_sequences=True),
keras.layers.TimeDistributed(keras.layers.Dense(24))
])

history = model6.fit(X_train, Y_train, epochs=10,batch_size=64,
validation_data=(X_valid, Y_valid))


So in this case 168 hours of the past are used (n_steps) to make a prediction for the next 24 hours of electricity prices. 6 features are used.

I have problems both understanding the format and the syntax for creating the inputdata of the RNN.

Format question

I uploaded a screenshot of the dimensions of the data-arrays from Spyder. So basically we have the full data array 'series_reshaped' with the size (61134, 192, 6). Then we have the input data X_train with the size (43800, 168, 6). The first dimension are the timeslot, the second dimension are the past timeslots that are used for prediction and the third timension are the 6 features for every of the 168 past timeslots. Then we have the labels Y_train with the size (43800, 168, 24). Here I do not understand why we have 168 in the second dimension. As far as I understood for each of the 168 past values * 6 features of the input data, we have 24 target values. So why is the second dimension then not 168*6 = 1008? Because we have a mapping of 1008 inputs to 24 outputs?

Syntax question

I do not really understand how these lines work in Python:

for step_ahead in range(1, 24 + 1):


Why does this create an Y array of the dimension (61134, 168, 24) or transfer the correct data into it? The index step_ahead only takes values from 1 to 24 and now we assign to 24 entries of the second dimension of the array Y 168 values from the past values of the series_reshaped. So why do we only assign the values to the 24 entries of the second dimension of Y and not to the full 168 entries? And why are we looking into the past data of the series_reshaped array (second dimension). For me these lines are extremely confusing altough they apparently do the right thing. Can anyone tell me a little bit more about the syntax of those lines?

Generally, I'd appreciate every comment and would be quite thankful for your help.

Update

Related questions: Hi all, as I still have problems with those question I would like to ask some related questions

1. About the creation of the input data: how can I know which structure the input data should have? And how can I then derive something like this code
for step_ahead in range(1, 24 + 1):


2)At the end of the training in the tutorial they use the following code for the prediciton

Y_pred = model6.predict(X_test)

last_list=[]

for i in range (0, len(Y_pred)):
last_list.append((Y_pred[i][0][23]))


So they take Y_pred[i][0][23] to construct the 1-dimensional list with the predicted values. Why do they take [0][23] and not for example [1][14]? They want to predict 24 hours in advance. Can I just always take Y_pred[i][0][23] ?

1. I still do not understand one of my inital questions: Why is the label dataset Y for training [Batch, 168, 24] if return sequence =true? We use the past 168 values to forecast 24 hours. We use 168*6 features for forecasting. For each element in the batch (each timeslot) we then have an output of 24 hours. So we should have the training data with dimension [Batch, 24] and not [Batch, 168, 24]. For every timeslot in the batch we need 168 past values. How is it then possible to map 24 hours of predictions to every 168 of the past values?

Reminder: My bountry expires in three days and unfortunately I have not received another more comprehensive answer. I'd highly appreciate any new answer that might explain the input data for time series forecasting with a recurrent neural network.

You should look at the data with Features intact for each step.Featues can't be flattened since each point of time is defined by all the Features.
Let's see this snap,

The upper table is the data
Let's assume we want to predict 2 steps using 3 input steps.

So, one instance of our input will have 3 sequential steps having 6 Features part of each sequential steps. So, input becomes [Batch, 3, 6]. In you case [Batch, 168, 6]

Output need to have 2 sequential steps(per requirement). Since we are predicting one feature, so it will have just one feature. So, output shape [Batch, 2]. In your case [Batch, 24].
But this would have been the case when we only want the backprop after the last step i.e.
return_sequences=False for the last RNN.
Since we are returning sequence every time, so output must have 24 values for each sequential step. So output becomes [Batch, 168, 24]

Features

Features don't pass sequentially, all 6 Features will pass together for each time step. Check this depiction below.
Each feature will go into each Neuron and each neuron will add a recurrent weight to each Neuron. If you check your model's parameter count for the first layer, it will be 540.
6*20(Input weights) + 20*20(Recurrent) + 20(Bias) = 540

$$\hspace{5cm}$$ Image credit - SO Answer

• Thanks 10xAI for your answer. Basically I understand the part until [Batch, 24] assuming you mean by batch just the whole timesequence. From then on I do not understand anymore. Why is the output size [Batch, 168, 24]? We use 168 time slots as the training data with 6 input features each. And this whole 'chunk' (having 168 * 6 = 1008 values) should be mapped to 24 target values. So I still think that the second dimension should be like [Batch, 1008, 24]. For each timestep in the batch, we have 1008 inputvalues to be mapped to 24 outputvalues. – PeterBe Mar 31 at 15:39
• I would also understand it if it was just [Batch, 24]. Then we could read it like this: For each of the timesteps in the batch, we have 24 outputvalues. The input data already contains the 168 *6 = 1008 features per timeslot [Batch, 168, 6]. So in this case the RNN would map the 1008 input features to the [Batch, 24] output values. – PeterBe Mar 31 at 15:42
• But I do not understand [Batch, 168, 24]. What is really done here? How can you read this. For every timestep in the batch we have 168 past values and we map them to 24 output values? But basically the mapping is not between the 168 pastvalues themselves and the 24 output values. It is between the 168*6 = 1008 features and the 24 output values. This is really confusing to me. – PeterBe Mar 31 at 15:45
• I stated in the beginning that please don't multiply 168*6. Every time-step will comprise 6 features. Features will be passed just the way we do in a simple NN. Added an edit in the answer. Now to the second point, we passed the first-time step, it will reach the output because the last RNN has return_sequences=True. So, we need the 24 values of the output. For the second time-step, the same thing will happen, so again we will need 24 values of the output...so on for 168 steps. So outputs need 24 values for each of 168 steps. – 10xAI Apr 1 at 12:51
• Thanks 10xAI for your answer and effort. I really appreciate it. You wrote "we passed the first-time step, it will reach the output because the last RNN has return_sequences=True. So, we need the 24 values of the output." Why will it reach the output and why do we then need 24 values of the output? And why do we need it for 168 steps? Basically we have a mapping of 168 *6 features ot 24 outputs as far as I understand because in the training data we use 168 past values with all its 6 features and this is done for every timeslot. – PeterBe Apr 1 at 14:59