What is the output of multivariate LSTM model?

Question

I am currently trying to build an LSTM model by using multivariate inputs, but I don't understand what exact output I am predicting.

I am currently using 5 features in the data as input data:

X_data = data[['Time', 'Avg CPU Load', 'F1', 'F2', 'F3']]

My main goal is to predict 'Avg CPU load' by using the other features as input data, but I am not sure if I am doing that.

In these features, I have made 'Time' as an index in the data frame. Then I have used 60 timesteps in the below code.

new_X_train=[]
new_Y_train=[]
for i in range(len(X_data)-timesteps-1):
    t=[]
    for j in range(0,timesteps):
        
        t.append(X_data[[(i+j)], :])
    new_X_train.append(t)
    new_Y_train.append(X_data[i+ timesteps,1])
 

The variables 'new_X_train' and 'new_Y_train' have the inputs data that have to be inserted into the model. Then I define my model which is.

sq = Sequential()
sq.add(LSTM(units = 100, return_sequences = True,
           input_shape = (None, 4)))
sq.add(Dropout(0.2))

sq.add(LSTM(units = 100,return_sequences = True))
sq.add(Dropout(0.2))

sq.add(LSTM(units = 100,return_sequences = True))
sq.add(Dropout(0.2))

sq.add(LSTM(units = 100))
sq.add(Dropout(0.2))

sq.add(Dense(units=1)) # only one output

sq.compile(optimizer = 'adam', loss = 'mean_squared_error')

sq.fit(new_X_train, new_Y_train, epochs = 5, batch_size = 20)

Once the model is trained, I load the test data and implement 60 timesteps. I.E: I do the same for what I have done for the training data, and then I use the X_test data to predict the values.

predicted = sq.predict(X_test)

Now I am not sure what output the above code is predicting, like what feature is it trying to predict?

And how to make my model predict the 'Avg CPU load' feature?

Answer 1

If you want to predict "Avg CPU Load", your y_train should be only the "Avg CPU Load" column and your x_train should be the rest of the columns (you might skip the time column if all timestamps are equally spaced).

The network will then learn a function mapping rest of the columns to "Avg CPU Load" and it shall be predicting the same.

In the testing phase, you also have to follow the same structure for x_test (and y_test i.e. ground truth).

In this case, for RNN, you are right by generating each training sample as all columns over 60 timestamps. You can check if the prediction is indeed what you want by plotting against the ground truth. Although, I think you should predict all the columns (new_y_train is all columns instead of avg CPU load only) i.e. make the output dimension the same as the input dimension.

Make the network predict the whole thing and pick your column of interest for your consideration. This is probably the more natural way to do it.