I am training an RNN on the following task: Given a sequence of thirty words, predict the next word.
Is there a benefit to having more than 30 cells (LSTM, GRU or plain RNN) in my network?
I've seen many examples online where similar networks are trained with multiple layers that each have 100 cells, but this does not make sense to me.
How does it help to have more cells than the length of the sequence? (in my case this length is 30)
I'm confused because from my understanding, each cell takes in two inputs
1. A new element of the sequence
2. The output from the previous cell
So after 30 cells, there will be no new sequence elements to input into the cell. Each cell will just be processing the output of the previous cell (receiving no new info).
I am using LSTM cells for this task (however, I'm guessing the actual type of RNN cell used is irrelevant).