It is entirely correct to apply PCA to a dataset like MNIST. Intuitively, corner pixels should almost never contain any information as to what digit is contained in the center of the image. So we should disregard them. You should expect similar results as with other datasets. PCA lowers the dimensionality of your data, thus allowing for a less complex model, however, this is at the cost of some information that is rejected when retaining only $n$ components.
When data is limited a less complex model will result in a lower bias and thus better accuracy when applied to novel data. However, the MNIST dataset is plentiful, thus the benefit of removing some features and losing even minimal information may cause your performance to degrade.