So I have ResNet50 trained to classify images.
For each prediction I track the time needed for it (input and model are moved to GPU):
start = time.time()
result = self.model.forward(transformed_image)
end = time.time()
print(end - start)
And always I get the following output:
1.0592937469482422
0.05996203422546387
0.06096029281616211
0.04996800422668457
So the first prediction is ~20 times longer than the following ones.
Why? And what happens behind the scenes when we launch prediction for the first time, using Torch?