# Why do RNNs share weight?

If weights are not shared the number of parameters will be extremely large and difficult to compute which I understand. I don't understand the argument that varying length inputs are taken care of by sharing weights as stated in many StackExchange answers like https://stats.stackexchange.com/questions/221513/why-are-the-weights-of-rnn-lstm-networks-shared-across-time or in this blog https://towardsdatascience.com/recurrent-neural-networks-d4642c9bc7ce. If in the architecture below I use different $$W_{e}^{(t)}$$ for each word at time $$t$$ then all of the $$W_{e}^{(t)}$$ will still have the same dimension because the embedding dimension for each word is same (every $$e^{(t)}$$ has the same size). And if we similarly take different $$W_{h}^{(t)}$$ at every time step (assuming all hidden states have the same number of nodes) then all $$W_{h}^{(t)}$$ will also have same dimensions. It will be equivalent to a series of vanilla NN's (inputs are embedding vector and previously hidden state vector of let's say dimension $$e$$ and $$h$$ respectively). Then the vanilla NN at $$t$$ time will output : $$h^{(t)}=\sigma(W_{h}^{(t)}h^{(t-1)}+W_{e}^{(t)}e^{(t)}+bias)$$
So how does using the same $$W_{h}$$ and $$W_{e}$$ solve the problem of variable input sequence lengths?

Also, I know that in standard RNNs like below hidden state kind of store the context from previous time steps so what is the interpretation of $$W_{h}$$ here?

• Can an RNN (by definition )simply not share weights between time-shifted inputs? At some point or another weights have to be shared simply because of the loops or time delays involved Jun 19 at 12:44