I'm in the beginning to learn and understand recurrent neural networks. As far as I can imagine, its multiple feed-forward neural networks with one neuron at each layer put next to each other, and connected from left to right, where each neuron is connected not just with the neuron below it, but the one at the left from the previous time. Not sure if it's a right way to think about it, but so far it's my first impression.

Some things are unclear though.

1. As far as I understood the final output of each timestep is supposed to predict the input of the next timestep. Is this true? What if I would just like to show the network two images of for example a horse, and depending on them, predict what distance did it go, and in which direction? Is this possible?

2. In the illustration above there's $A_0$. From where? I would assume at least two timesteps are needed to make a prediction, so in my understanding an $x_0$ is missing from the left side of the diagram. Am I right?

3. I've been reading through an article which says "Lets train a 2-layer LSTM with 512 hidden nodes". Does it mean two layer of activations, and 512 timesteps?