Also the paper said, we do not have a complete explanation to this phenomenon.
But here is kinda explanation about that.
While we do not have a complete explanation to this phenomenon, we
believe that it is caused by the introduction of many short term
dependencies to the dataset.
Normally, when we concatenate a source sentence with a target
sentence, each word in the source sentence is far from its
corresponding word in the target sentence.
As a result, the problem has a large “minimal time lag” .
By reversing the words in the source sentence, the average distance
between corresponding words in the source and target language is
However, the first few words in the source language are now very close
to the first few words in the target language, so the problem’s
minimal time lag is greatly reduced.
Thus, backpropagation has an easier time “establishing communication”
between the source sentence and the target sentence, which in turn
results in substantially improved overall performance.