So I have ResNet50 trained to classify images.
For each prediction I track the time needed for it (input and model are moved to GPU):
start = time.time() result = self.model.forward(transformed_image) end = time.time() print(end - start)
And always I get the following output:
1.0592937469482422 0.05996203422546387 0.06096029281616211 0.04996800422668457
So the first prediction is ~20 times longer than the following ones.
Why? And what happens behind the scenes when we launch prediction for the first time, using Torch?