I'm late for this post. Let's break down the problem into *manageable* parts and address each question. I'll provide insights into the format, syntax, and logic of your code.
1. Format Question: Understanding Dimensions
> *Why is `Y_train` shaped `(43800, 168, 24)` and not `(43800, 1008, 24)`?*
The confusion arises from a misunderstanding of how the **input** data (`X_train`) and **target labels** (`Y_train`) are structured. Here's the breakdown:
- **input** data `X_train` `(43800, 168, 6)`:
Each sample in the batch (`43800` timeslots) consists of 168 time
steps (`n_steps`) of historical data.
For each of these time steps, 6 features (e.g., temperature, wind speed, etc.)
are provided.
- **target labels** `Y_train` `(43800, 168, 24)`:
Here, the second dimension (`168`) aligns with the 168 time steps of
`X_train`. For each of these 168 time steps, we predict 24 future values (for
each time step, we aim to forecast the next 24 hours).
The key point is that the **RNN is configured to predict 24 future steps for
each of the 168 input time steps**. This explains why `Y_train` has the shape
`(43800, 168, 24)`.
If we wanted the model to predict only 24 future values for the entire 168-step input sequence (rather than for each time step within the sequence), then `Y_train` would have been `(43800, 24)`.
---
2. Syntax Question: Code Analysis
> for step_ahead in range(1, 24 + 1):
> Y[..., step_ahead - 1] = series_reshaped[..., step_ahead:step_ahead + n_steps, 0]
Let's break it down step-by-step:
- `range(1, 24 + 1)`:
- Loops from `1` to `24` (inclusive), where each value represents the future step to predict (e.g., `1 hour ahead`, `2 hours ahead`, ..., `24 hours ahead`).
- `series_reshaped[..., step_ahead:step_ahead + n_steps, 0]`: This slices the `series_reshaped` array:
- `step_ahead` determines the starting point of the slice.
- `step_ahead + n_steps` defines the endpoint of the slice (168 steps ahead).
- `0` selects the feature at index 0 (assumed to be the target variable, e.g., electricity price).
- `Y[..., step_ahead - 1]` assigns the sliced data to the correct position in the `Y` array:
- `step_ahead - 1` ensures that each prediction (1-hour ahead, 2-hours ahead, etc.) is stored in a separate "slot" along the last dimension of `Y`.
> How does this create `Y` with the correct dimensions `(61134, 168, 24)`?
The outermost loop iterates over the `24` future steps, filling the last dimension of `Y` one step at a time.
For each future step, `168` time steps of historical data are used to generate the prediction.
---
- Predicting Future Values
> *Why do we use `Y_pred[i][0][23]` in the prediction code?*
> Y_pred = model6.predict(X_test)
> last_list = []
> for i in range(len(Y_pred)):
> last_list.append(Y_pred[i][0][23])
- `Y_pred[i][0][23]`:
- `i`: Refers to the i-th sample in the batch of predictions.
- `0`: Refers to the first time step within the sequence (the initial context for prediction).
- `23`: Selects the 24th prediction (corresponding to 24 hours ahead).
This selection makes sense because you are interested in the **24-hour-ahead prediction for the first time step** of each input sequence.
---
4. > *Why is `Y` shaped `(Batch, 168, 24)` instead of `(Batch, 24)`?*
This is because the RNN is configured with `return_sequences=True`. Here’s what it means:
- `return_sequences=True`: The RNN outputs a prediction for every time step in the input sequence (`168` steps).
- For each of these `168` steps, the model predicts the next `24` steps into the future, resulting in a target shape of `(Batch, 168, 24)`.
If the model was configured with `return_sequences=False`, it would only produce a single prediction for the entire input sequence (likely `(Batch, 24)`).
||
|:--:|
| Fig. 1: Data[Time][X][y] [pic credit](https://towardsdatascience.com/recurrent-neural-networks-for-electricity-price-prediction-a26f8411ea44)|
---
5. General Principles for Structuring Input Data
> *How to Determine the Input Data Structure?*
The input data structure depends on the problem and the model configuration:
- **Time Steps** (`n_steps`): The length of the historical data sequence used for predictions.
- **Features**: Number of features available at each time step.
- **Target** (`Y`): Number of future steps you want the model to predict.
For example:
- For a task predicting **1-step-ahead**: Input could be `(Batch, n_steps, Features)` and target could be `(Batch, 1)`.
- For **multi-step forecasting (e.g., 24 steps)**: Input remains `(Batch, n_steps, Features)` but target expands to `(Batch, 24)` (or `(Batch, n_steps, 24)` if predicting for every time step).
---
6. Predicting Outputs at Specific Time Steps
> *Can you always take `Y_pred[i][0][23]`?*
Yes, if your goal is to predict 24 hours ahead for the first time step (`0`) of each input sequence, you can always extract `Y_pred[i][0][23]`.
If you are interested in predictions for other time steps, you can modify the indices accordingly:
- Example: `Y_pred[i][5][23]` would give the 24-hour-ahead prediction for the 6th time step in the sequence.
---
Based on the [link](https://towardsdatascience.com/recurrent-neural-networks-for-electricity-price-prediction-a26f8411ea44) you shared *"Recurrent Neural Networks for Electricity Price Prediction"*:
[![enter image description here][1]][1]
Following code is working and applies [`SimpleRNN()`](https://keras.io/api/layers/recurrent_layers/simple_rnn/) class on data provider cleaned and shared in his [Github](https://raw.githubusercontent.com/Carterbouley/ElectricityPricePrediction/refs/heads/master/re_fixed_multivariate_timeseires.csv) repo:
```python
# Import necessary libraries
import pandas as pd
import numpy as np
import tensorflow as tf
from tensorflow import keras
from matplotlib import pyplot as plt
# Load the dataset
# Replace the file path with the one you download from the GitHub repository.
# Step 1: Load the dataset
# Read the data for the parameters from the GitHub URL into a Pandas DataFrame
url = "https://raw.githubusercontent.com/Carterbouley/ElectricityPricePrediction/refs/heads/master/re_fixed_multivariate_timeseires.csv"
# Fetch and load the CSV data
df = pd.read_csv(url, sep=",", low_memory=False)
print(df.columns) #['datetime', 'GBP/mWh', 'temperature', 'coal Price', 'oil Price', 'uranium Price','natural gas Price']
# Drop the 'datetime' column since it's not needed for the forecasting
del df['datetime']
# Convert the dataframe to a NumPy array for easier processing
data = df.values
# Define the number of past time steps used for forecasting
n_steps = 168 # One week's worth of hourly data (7 days * 24 hours)
# Reshape the data to create sequences for time series forecasting
# Each sequence includes `n_steps + 24` steps (past data + future target)
series_reshaped = np.array([
data[i:i + (n_steps + 24)].copy()
for i in range(len(data) - (n_steps + 24))
])
print(series_reshaped.shape) #(61134, 192, 6)
# Split the sequences into training, validation, and test datasets
X_train = series_reshaped[:43800, :n_steps] # First 43800 samples for training
X_valid = series_reshaped[43800:52560, :n_steps] # Next 8760 samples for validation
X_test = series_reshaped[52560:, :n_steps] # Remaining samples for testing
# Create the target variable `Y` for forecasting
# Target consists of 24 future steps for each sequence
Y = np.empty((series_reshaped.shape[0], n_steps, 24)) # Pre-allocate target array
for step_ahead in range(1, 24 + 1): # Loop over each future step
# Shift the target by `step_ahead` to align it with the input sequence
Y[..., step_ahead - 1] = series_reshaped[..., step_ahead:step_ahead + n_steps, 0]
# Split the target variable into training, validation, and test sets
Y_train = Y[:43800] # Training target
Y_valid = Y[43800:52560] # Validation target
Y_test = Y[52560:] # Testing target
# Set random seeds for reproducibility
np.random.seed(42)
tf.random.set_seed(42)
# Define the model architecture
model6 = keras.models.Sequential([
keras.layers.SimpleRNN(20, return_sequences=True, input_shape=[None, 6]), # First RNN layer
keras.layers.SimpleRNN(20, return_sequences=True), # Second RNN layer
keras.layers.TimeDistributed(keras.layers.Dense(24)) # Dense layer applied at each time step
])
# Compile the model with appropriate loss function and optimizer
model6.compile(
loss="mean_squared_error", # Mean squared error loss for regression tasks
optimizer="adam", # Adam optimizer for efficient training
metrics=['mean_absolute_percentage_error'] # Evaluation metric
)
# Train the model on the training data
history = model6.fit(
X_train, Y_train, # Training input and target
epochs=10, # Number of epochs
batch_size=64, # Batch size
validation_data=(X_valid, Y_valid) # Validation data
)
print(model6.summary())
#Model: "sequential"
#┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
#┃ Layer (type) ┃ Output Shape ┃ Param # ┃
#┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
#│ simple_rnn (SimpleRNN) │ (None, None, 20) │ 540 │
#├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
#│ simple_rnn_1 (SimpleRNN) │ (None, None, 20) │ 820 │
#├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
#│ time_distributed (TimeDistributed) │ (None, None, 24) │ 504 │
#└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
# Total params: 5,594 (21.86 KB)
# Trainable params: 1,864 (7.28 KB)
# Non-trainable params: 0 (0.00 B)
# Optimizer params: 3,730 (14.57 KB)
```
[![enter image description here][2]][2]
[1]: https://i.sstatic.net/xYyK4ZiI.png
[2]: https://i.sstatic.net/ZIzVXXmS.png