Any difference between adding epochs and duplicating data for neural nets?

Question

Let's say I am training a neural net (e.g. convolutional network or LSTM).

Generally, the longer the training (more epochs) leads to better accuracy, albeit at times at the expense of overfitting.

Another approach is to duplicate the data. Again, if I have 1000 examples, I could duplicate them 10 times and randomise them, and the neural net will learn better (but again, it may not be able to generalise well).

Is there any difference between duplicating data and adding epochs when training a neural net in terms of its learning?

Answer 1

The most obvious difference is that by adding epochs, you (still) never get the same observation twice in a same batch, which isn't the case when you duplicate data.

I'm no expert in NNs, but it seems that if you do not introduce any noise in the duplicated data, then you might as well just add epochs.

Of course, if you fit two models over a single epoch,

model 1: using your n=1000 observations

model 2: using your n*10 = 10,000 observations (with duplications)

your second model will most probably return a lower loss value. However, a fairer comparison would be to train model 1 with 10 epochs so that in both cases you expose your net to your observations the same number of times in total.