Dealing with inputs of different sizes in time series forecasting

Question

I'm dealing with a task where I need to forecast the n-ith value of a target variable in a multivariate time series. But in this case we have two variables: -var1: Is my target variable that represents the output of a system. -var2: This time series represents a binary control signal (on/off).

Thus, var1 varies according to its previous values and according to the control signal (var2).

My problem is that the output in a given day n depends on the last n values of var 2 and the last n-1 values of var 1. That is, I have a different number of values (n and n-1) as the input of my network.

In this scenario, I'm not sure about how to model the input of my network.

I was trying to use a LSTM layer as the input of my network, with 2 dimensions (var1 and var2). But, as I said, each "sample" has n values of var2 and n-1 values of var1. It is not possible to create a 2D array in this situation.

Any idea?

Answer 1

You can create a sort of encoder-decoder network with two different inputs.

latent_dim = 16

# First branch of the net is an lstm which finds an embedding for the var1
var_1_inputs = tf.keras.Input(shape=(window_len_1, n_1_features), name='var_1_inputs')
# Encoding var_1
encoder = tf.keras.layers.LSTM(latent_dim, return_state=True, name = 'Encoder')
encoder_outputs, state_h, state_c = encoder(var_1_inputs) # Apply the encoder object to var_1_inputs.

var_2_inputs = tf.keras.Input(shape=(window_len_2, n_2_features), name='var_2_inputs')
# Combining future inputs with recurrent branch output
decoder_lstm = tf.keras.layers.LSTM(latent_dim, return_sequences=True, name = 'Decoder')
x = decoder_lstm(var_2_inputs, 
                               initial_state=[state_h, state_c])

x = tf.keras.layers.Dense(16, activation='relu')(x)
x = tf.keras.layers.Dense(16, activation='relu')(x)
output = tf.keras.layers.Dense(1, activation='relu')(x)

model = tf.keras.models.Model(inputs=[var_1_inputs,var_2_inputs], outputs=output)

optimizer = tf.keras.optimizers.Adam()
loss = tf.keras.losses.Huber()
model.compile(loss=loss, optimizer=optimizer, metrics=["mae"])

model.summary()

Here you are, of course I inserted random numbers for layer, latent dimensions, etc.

You can also have different features to input with var_1 and var_2 and these have to passed as arrays.