I used pytorch as the training framework and the official pytorch imagenet example to train the image classification model with my custom dataset.
My custom dataset has 2 different label (good and bad), and over 1 million images.
I splitted the dataset into a training set(80%), a val set(10%), and a test set(10%)
My model got average 99% training acc in training phase, and nearly 99% val acc in validation phase. In the testing phase, the model got 99% testing acc.
However, when I used my model to evaluate the whole dataset(all the images in my dataset), the acc got only 90%, which is pretty weird since my model updated its parameter in the training phase.
The model should be able to achieve higher accuracy, but it can only get 90% acc when evaluating the whole dataset.
I am wondering if it is normal or anything I can check for this problem.