Input Data for LSTM RNN transformation

I am new to RNN, and I do not know how to transform my input data in order to use it in a LSTM layer.

Currently, my input data has (1000 samples, 50 features). But let´s say it is only (1000 samples, 2 features) to make the example below easier. Each sample would look like this:

[Date, Score, team_id]
1.[11/10/2022, 56, 23543]
2.[12/10/2022, 75, 44354]
3.[17/10/2022, 56, 44354]....


I would like to use 5 time steps, so my data will have a shape like this (1000, 5, 50). I know I would need to get the 5 previous scores for all samples, but once I do that, I do not understand how to represent the data. I guess it will be like a list of matrices, but how is that coded from the dataframe I have?

UPDATE! I have created the list of matrices where if you print(list[0]), you get an array of dicts with the information, so we could say it is a list of lists of dicts. How do I convert that into a dataframe or input data?

Thank you very much

You can use the available function Keras or any other library though not very commonly available.
This will give you a lot of functionalities out of the box. tf link

import pandas as pd
from tensorflow.keras.preprocessing import timeseries_dataset_from_array

seq_len=5
tf_x = timeseries_dataset_from_array(
data, None, sequence_length=seq_len, batch_size=32, end_index=data.shape[0]-seq_len )
for x in tf_x.take(1):
print(x.shape)
print("Data - ",x[0,0])


You can create your own using Pandas shift and Numpy concatenate.
Create n(seq_len) array of the initial data frame by shifting. Then concatenate all of them together on 1st axis.

import numpy as np
seq_len=5

list_np_arr=[]
for i in range(seq_len):
list_np_arr.append(data.shift(i+1).to_numpy()[:,np.newaxis,:])
# shape = # df_size-2*seq_len, seq_len, n_features
np.concatenate((list_np_arr), axis=1)[seq_len:-seq_len,:,:].shape


Slicing (i.e. seq_len:-seq_len) was done to remove the initial and ending portion as it will have NaN of the shifted data.

• Hello, thanks for your answer! What type of variable 'data' is in your code? Because in my program it is a list, and it does not support some of the methods like shape or shift... Oct 26, 2022 at 8:22