I'm new to Recurrent Neural Network but I want to train my data with LSTM but I'm having a trouble to understand LSTM Many-to-One architecture.
Suppose the size of my data is
time_step x num_features say
2 x 2 and I have to use many-to-one LSTM architecture because I want to do classification. So in the last time_step I have to add dense (a) which contain sigmoid activation function to predict the sequence class which is 0 or 1.
My questions are,
- When I compute a, do I need to include all the hidden state (h1 and h2) or just the last hidden state h2?
- If I just include the h2, how do I calculate the derivatives of the loss function (cross entropy) w.r.t h1?
The derivation for h1 is highly recommended. Thank you :)