I'm trying to use CNN for time series regression in python. I have 9 elements in each time step (from sensor readings) and the output (target/reference) is 4 elements.
Input Shape = (time steps, 9)
Output Shape = (time steps, 4)
Based on papers I should use rolling windows, such as:
I don't understand how could I implement that. Should I convert the input to as follows?
Input Shape = (Time Steps, Sliding Windows Length, 9)
The Model is:
####################################################################################################################
# Define ANN Model
# define two sets of inputs
acc = layers.Input(shape=(3,1,))
gyro = layers.Input(shape=(3,1,))
# the first branch operates on the first input
x = Conv1D(256, 1, activation='relu')(acc)
x = Conv1D(128, 1, activation='relu')(x)
x = Conv1D(64, 1, activation='relu')(x)
x = MaxPooling1D(pool_size=3)(x)
x = Model(inputs=acc, outputs=x)
# the second branch opreates on the second input
y = Conv1D(256, 1, activation='relu')(gyro)
y = Conv1D(128, 1, activation='relu')(y)
y = Conv1D(64, 1, activation='relu')(y)
y = MaxPooling1D(pool_size=3)(y)
y = Model(inputs=gyro, outputs=y)
# combine the output of the three branches
combined = layers.concatenate([x.output, y.output])
# combined outputs
z = Bidirectional(LSTM(128, dropout=0.25, return_sequences=False,activation='tanh'))(combined)
z = Reshape((256,1),input_shape=(128,))
z = Bidirectional(LSTM(128, dropout=0.25, return_sequences=False,activation='tanh'))(combined)
#z = Dense(10, activation="relu")(z)
z = Flatten()(z)
z = Dense(4, activation="linear")(z)
model = Model(inputs=[x.input, y.input], outputs=z)
model.compile(loss='mse', optimizer = tf.keras.optimizers.Adam(learning_rate=0.01),metrics=['accuracy','mse'],run_eagerly=True) #, callbacks=[tensorboard]
model.summary()
Model: "model_2"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 3, 1)] 0 []
input_2 (InputLayer) [(None, 3, 1)] 0 []
conv1d (Conv1D) (None, 3, 256) 512 ['input_1[0][0]']
conv1d_3 (Conv1D) (None, 3, 256) 512 ['input_2[0][0]']
conv1d_1 (Conv1D) (None, 3, 128) 32896 ['conv1d[0][0]']
conv1d_4 (Conv1D) (None, 3, 128) 32896 ['conv1d_3[0][0]']
conv1d_2 (Conv1D) (None, 3, 64) 8256 ['conv1d_1[0][0]']
conv1d_5 (Conv1D) (None, 3, 64) 8256 ['conv1d_4[0][0]']
max_pooling1d (MaxPooling1D) (None, 1, 64) 0 ['conv1d_2[0][0]']
max_pooling1d_1 (MaxPooling1D) (None, 1, 64) 0 ['conv1d_5[0][0]']
concatenate (Concatenate) (None, 1, 128) 0 ['max_pooling1d[0][0]',
'max_pooling1d_1[0][0]']
bidirectional_1 (Bidirectional (None, 256) 263168 ['concatenate[0][0]']
)
flatten (Flatten) (None, 256) 0 ['bidirectional_1[0][0]']
dense (Dense) (None, 4) 1028 ['flatten[0][0]']
==================================================================================================
Total params: 347,524
Trainable params: 347,524
Non-trainable params: 0