0
$\begingroup$

I have been learning keras and TensorFlow for some weeks now, and get confused with epoch.

I trained my network for 50 epochs, the test data and training data are randomly split (80% train, 20% test). The training data's accuracy grows nicely, but the test data's accuracy goes up and down. I am not sure why it happens like that.

Say in epoch 10, the test data's accuracy is 92%, in next epoch, how can accuracy drop? The thing I can think of is that maybe for each epoch, the test data and training data are randomly selected again, so the system sees new data which doesn't fit previous parameters?

Is it the case?

enter image description here

$\endgroup$
2
$\begingroup$

Variation in test data accuracy is perfectly normal. Your network is fitting for your train data. If you see (big) variation in train accuracy, then you should be worried about learning rate tuning, regularization etc.
But here, everything seems to be going fine. Remember your network only fits on your train data, so it modifies its weights according to that data. Since your test data -which the network never uses for weight update- is not exactly your train data (even though it should come from the same distribution ... so test accuracy should globally improve over iterations if the network is learning), test accuracy is bound not to improve exactly at each iteration.

$\endgroup$
2
  • $\begingroup$ So for the 80% training data and 20%test data, it is something fixed at the beginning, so every epoch uses the same training and test data? $\endgroup$ – daxu Jul 26 '18 at 8:24
  • 1
    $\begingroup$ In general (in "vanilla" implementations of neural network training), yes. Now if some fancier stuff like cross-validation of testing set is going on, it might be more complicated... You should be able to figure that out looking at your code ! $\endgroup$ – Soltius Jul 26 '18 at 8:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.