Confusion regarding epoch and accuracy

Question

I have been learning keras and TensorFlow for some weeks now, and get confused with epoch.

I trained my network for 50 epochs, the test data and training data are randomly split (80% train, 20% test). The training data's accuracy grows nicely, but the test data's accuracy goes up and down. I am not sure why it happens like that.

Say in epoch 10, the test data's accuracy is 92%, in next epoch, how can accuracy drop? The thing I can think of is that maybe for each epoch, the test data and training data are randomly selected again, so the system sees new data which doesn't fit previous parameters?

Is it the case?

Answer 1

Variation in test data accuracy is perfectly normal. Your network is fitting for your train data. If you see (big) variation in train accuracy, then you should be worried about learning rate tuning, regularization etc.
But here, everything seems to be going fine. Remember your network only fits on your train data, so it modifies its weights according to that data. Since your test data -which the network never uses for weight update- is not exactly your train data (even though it should come from the same distribution ... so test accuracy should globally improve over iterations if the network is learning), test accuracy is bound not to improve exactly at each iteration.