Dataset image size and inference speed

Question

Does training/fine tuning a pre-trained model on a the same dataset but with sizes scaled down (e.g., by 70%) improve inference speed? More generally, does training a CNN on smaller images improve inference speed?

Answer 1

As you have phrased it the answer is no.

Models typically have a fixed input and output size (except RNNs, which are not really used for images). This means that you can not input smaller images to the network. If it appears that you are doing so it is either because the network is automatically resizing or padding your small image up to the model input size. In these cases, the time performance of the model will be fairly constant.

If you were to train the same model with a smaller image input size, you would be correct in saying that inference (and train) time would be faster than the larger image input size model. You could also load the pre-trained weights from the larger model into the smaller model, in most cases.

Note that what you are considering will change the input resolution and as such the features will change, decreasing the loss-performance of the model.

Answer 2

Yes - Smaller images means smaller matrices to multiply and will decrease prediction time. The exact amount is an empirical question which could be answered through benchmarking.

However, the speed improvement might not meaningful. There are many ways to improve prediction speed (e.g., switch architectures, distillation, and reduced precision) which would have larger effect than smaller image size.