MNIST dataset with 60 000 training samples and 10 000 test samples.
Neural network #1. Accuracy on the training set: 99.53%. Accuracy on the test set: 99.31%.
Neural network #2. Accuracy on the training set: 100.0%. Accuracy on the test set: 99.19%.
Which neural network is better if other parameters are unknown?
I have seen how many studies focus on accuracy on a test set, and rarely write about accuracy on a training set.
The first neural network is better in accuracy on the test set, but worse in accuracy on the training set.
Would you say that unlearned training samples can be bad for testing a different test set?
I have an idea to compare for overall accuracy:
(99.53% * 60000 + 99.31% * 10000) / (60000 + 10000) = 99.499%
(100.0% * 60000 + 99.19% * 10000) / (60000 + 10000) = 99.884%
Or it could be a weighted multiplication.
But I'm not sure about that.
What do you think about this?