As reference to Why should the data be shuffled for machine learning tasks I was curious if this is the case for neural-machine-translation.
I would like to analyze why I think this is the case for the 3 components (train, test, val) parts where the data (as a reference dataset think of the europarl-dataset) is split:
for the training part, when doing mini-batch gradient descent it is better that for each update we collect data that are closer semantically, so that the learning gets focused each time on a certain direction. For example is better to have a lot of topics for let's say coffee together for a single update, than topics about drugs or coffee, dogs etc in a single update
for the test part, we would like to see how the model behaves in a big text of continuous sentences, as the goal of translation is that of documents. The same should hold for validation
I would like your view on that if you have any empirical or theoretical insight of why this is not the case for neural machine translation.