Why does adding data augmentation decrease training accuracy a tiny bit?

Question

Before data augmentation, my model clearly overfits and hits a 100% training accuracy and a 52% validation accuracy. When only adding data augmentation with Keras, as a regularization technique, it achieves a 95% training accuracy with slower convergence and a 80% validation accuracy (which is a way better result). But why does the training accuracy gets reduced by around 5%?

If somebody could provide the link to a research paper or explain the reasoning behind this, it would be greatly appreciated!

Answer 1

The obvious reasons why data augmentation might reduce the train accuracy is -

As you know, deep learning models are data hungry. If the model don't get enough data to recognize the patterns then it will try to memorize the dataset. Bigger models tend to memorize the data instead of finding patterns, because they are big enough to do so. When model memorizes the training data it will definitely perform very good on training set and poorly on validation set.

And as you said data augmentation is a regularization technique. In regularization your model weights are penalized more to make sure they don't over fit. As a result, your model cannot perform well on training set (depending on how much regularzation is used), but as an advantage model will try to find generalized patterns in the dataset and this will also help at the time of validation.

I could find one research paper which has exhaustive experiments about data augmentation and regularization.