How many RNN units are needed for tasks involving sequences?

Question

I am training an RNN on the following task: Given a sequence of thirty words, predict the next word.

Is there a benefit to having more than 30 cells (LSTM, GRU or plain RNN) in my network?
I've seen many examples online where similar networks are trained with multiple layers that each have 100 cells, but this does not make sense to me.
How does it help to have more cells than the length of the sequence? (in my case this length is 30)

I'm confused because from my understanding, each cell takes in two inputs
1. A new element of the sequence
2. The output from the previous cell
So after 30 cells, there will be no new sequence elements to input into the cell. Each cell will just be processing the output of the previous cell (receiving no new info).

I am using LSTM cells for this task (however, I'm guessing the actual type of RNN cell used is irrelevant).

Answer 1

You don't need to assign an RNN cell to each entry in a sequence, you just need one.

A single cell can process sequences because of its recursive nature(its output feeds back into itself):

$$ h_t = f(h_{t-1}, W, x_t) $$

where $h_t$ is the hidden state at step $t$, $W$ is the weight used for the cell, and $x_t$ is the current input at step $t$ .

Hopefully, you can see how it is only necessary for us one weight matrix, $W$, instead of multiple as your understanding would imply.

Where you got the notion of multiple RNN cells is probably when we feed the hidden state as the input, $x$, into another RNN cell. More layers of RNN cells, mean you can learn a more complex sequence.

Conclusively, your RNN cell cell can't work without new inputs, therefore it end when your sequence ends. Also, it is recommended to use LSTM or GRU over vanilla RNN units.