2
$\begingroup$

So I have ResNet50 trained to classify images.

For each prediction I track the time needed for it (input and model are moved to GPU):

    start = time.time()
    result = self.model.forward(transformed_image)
    end = time.time()
    print(end - start)

And always I get the following output:

1.0592937469482422
0.05996203422546387
0.06096029281616211
0.04996800422668457

So the first prediction is ~20 times longer than the following ones.

Why? And what happens behind the scenes when we launch prediction for the first time, using Torch?

$\endgroup$

2 Answers 2

1
$\begingroup$

I have seen a similar question several times before. See https://stackoverflow.com/a/55577921/9212382 for a possible explanation.

$\endgroup$
1
$\begingroup$

This is due to the fact that first time is very important, everything is set up during the first pass like cache, memory on the GPU, graph optimization, etc. Also 1 seconds is not that long it can take few seconds just to run matrix multiplication on GPU

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.