It is usually better if you have a not so large but balanced dataset and you are performing classification to apply stratification in order to split it in a training and testing datasets which are both balanced as well.
So there is a notion of having the training and testing datasets represent adequately the overall dataset.
Could you expand this notion to regression?
So the idea is that instead of just shuffling and splitting the dataset, you could group the targets in bins for example [0, 10] [10, 20] etc. And then have the training and testing dataset be also an adequate representation of the whole dataset by having targets with all kinds of values. (Otherwise, you could end up leaving out some part of the range)
Makes sense? :)