# Training accuracy with SGD

So I am computing the training accuracy of my CNN on a batch training data and it almost always gives 100% accuracy. But I found from somewhere that in order to get the training accuracy, it is tested against the entire training data and not the specific batch that you used for your training.

Is training accuracy even computed? I wonder if you don't use batches for training. Then it will be 100% accuracy always?

I'm suspecting that this is due to using a small batch size.

• Accuracy can be too noisy to be evaluated on minibatches, although it is not always as bad as this. Look at the validation accuracy and training proxy loss (e.g., cross entropy).
– Emre
Jun 10 '17 at 17:42
• I just realized that I was using minibatch size 10 Jun 10 '17 at 17:46

Training accuracy is not considered because it can give falsified evaluation value for over-fitting 100% in your case. Hence it is always computed against the batch not entertained for training. This corresponds to the generalized error. For more information see K-Fold cross validation.