2
$\begingroup$

I want to train an LSTM network for time-series predictions, and want to get to the bottom of LSTM's. In my understanding, the number of cells in a single LSTM layer can vary. However, since each cell takes an input at time-step t, wouldn't the number of cells need to be equal to t?

For example (from TensorFlow tutorial):

 t=0  t=1    t=2  t=3     t=4
[The, brown, fox, is,     quick]
[The, red,   fox, jumped, high]

words_in_dataset[0] = [The, The]
words_in_dataset[1] = [brown, red]
words_in_dataset[2] = [fox, fox]
words_in_dataset[3] = [is, jumped]
words_in_dataset[4] = [quick, high]
batch_size = 2, time_steps = 5

Wouldn't the maximum number of cells in each layer be 5? since after the 5th input we no longer have any other information to input. However, I've seen many networks with a higher number of cells than that. Therefore, why is this a possibility?

$\endgroup$
1
$\begingroup$

Consider RNN networks as a simple MLP which for each time span t, takes the inputs of that time and the outputs of the previous step. Actually each time, you unroll the network. Consequently, the number of cells does not have any relation to the input size or the length of the time series. Take a look at here for a better understanding.

$\endgroup$
  • 1
    $\begingroup$ Thanks! Just like with MLP you can have two inputs x1 and x2 go to any number of hidden cells as well. I had gotten confused with the terminology. $\endgroup$ – Mati K Feb 12 at 19:12
1
$\begingroup$

No of cell doesn't have any relation with input length. You can simply see it as how many time you are passing the input through an abstraction function in order to obtain a higher level representation.

Cell no.----Input----output

1 . | your input | output-1

  1. | output-1 | output-2
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.