If I have virtually endless training data (it's synthesized) is there still purpose in having epochs? I.e. training on the same samples multiple times?
If the data is unlimited, how would you have an epoch to begin with? For example, if you are analyzing tweets, you could never finish an epoch will all the tweets, since there will be an endless supply of new tweets. A much better approach will be to do some online or streaming learning.
Would it make sense to create a subset by ignoring new incoming tweets or data in general?
That really depends on your problem. For example if you have a datastream of tweets, or any other where new data is being produced in real time, you would miss out any trends or patterns that emerged after you sampled from the stream. Are those missed patterns relevant to your problem? They may or may not be.