Hi I was working on a project to generate a hardware implementation of an LSTM network for anomaly detection. I was trying to understand the general structure of an lstm network for forecasting uni-variate time-series. I was working on creating a layer that in Keras would be implemented as
self.model.add(LSTM(units=num_neurons, return_sequences=False, batch_input_shape=(1, b - 1, 1), stateful=True)).
where b is the lookback factor
I was working off the generic image of an lstm cell where it takes in the inputs of x_t, c_t-1, and h_t-1. I think I'm confusing my self with all the different terminology and I was hoping someone could clarify and help me confirm my mental model of how this should all be working.
Starting off the num_neurons in the code, I'm assuming that is the number of lstm cells in layer. I'm assuming the input to each of these cells for the x_t term is the same vector that is a vector of [x, x-1,..,x-b]. I honestly have no clue what c_t-1 and h_t-1 are meant to be. I had two possible ideas. Idea 1 is that each lstm unit in same layer feeds into the next one. Idea 2 is that each lstm in same layer are not connected at all rather the layer is iterated through several times, like each cells output feeds back into itself several times. In either case, I don't know what the base case of h and c would be.
I found this one youtube video by StatQuest which honestly makes perfect sense, https://www.youtube.com/watch?v=YCzL96nL7j0. The lstm network described is a series of connected LSTM cells that each take in one value each from the vector [x_t, x_t-1, ... x_t-b] and this makes complete sense to me. But then I don't understand how in keras you can have a different number of neurons and a different input vector size.