4
$\begingroup$

I had been working with pre-trained models and was just curious to know the fastest forward propagating model of all the computer vision pre-trained models. I have been trying to achieve faster processing in one-shot learning and have tried the forward propagation with few models over a single image and the results are as follows:

  • VGG16: 4.857 seconds
  • ResNet50: 0.227 seconds
  • Inception: 0.135 seconds

Can you tell the fastest pre-trained model available out there and the drastic time consumption difference amongst the above-mentioned models.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
6
$\begingroup$

The answer will depend on some things such as your hardware and the image you process. Additional, we should distinguish if you are talking about a single run through the network in training mode or in inference mode. In the former, additional parameters are pre-computed and cached as well as several layers, such as dropout, being used, which are simply left out during inference. I will assume you want to simply produce a single prediction for a single image, so we are talking about inference time.

Factors

The basic correlation will be:

  • more parameters (i.e. learnable weights, bigger network) - slower than a model with less parameters
  • more recurrent units - slower than a convolutional network, which is slower than a full-connected network1
  • complicated activation functions - slower than simple ones, such as ReLU
  • deeper networks - slower than shallow networks (with same number of parameters) as less run in parallel on a GPU

Having listed a few factors in the final inference time required (time taken to produce one forward run through the network), I would guess that MobileNetV2 is probably among the fastest pre-trained model (available in Keras). We can see from the following table that this network has a small memory footprint of only 14 megabytes with ~3.5 million parameters. Compare that to your VGG test, with its ~138 million... 40 times more! In addition, the main workhorse layer of MobileNetV2 is a conv layer - they are essentially clever and smaller versions of residual networks.

Keras pre-trained models

Extra considerations

The reason I included the whole table above was to highlight that with small memory footprints and fast inference times, comes a cost: low accuracies!

If you compute the ratios of top-5 accuracy versus number of parameters (and generally versus memory), you might find a nice balance between inference time and performance.

1 Have a look at this comparison of CNNs with Recurrent modules

Improve this answer
$\endgroup$
2
  • $\begingroup$ It is not as simple as looking at the number of parameters and depth. For instance, ResNet50 is significantly faster than Xception on my hardware, despite having more parameters and a higher depth. Similarly, MobileNet is faster than MovileNetV2 for me. That being said, the mobile nets are effectively built for embedded hardware, and thus less demanding. $\endgroup$
    – Wok
    Commented Feb 8, 2019 at 13:02
  • $\begingroup$ @Wok - You're right, which is why I didn't say that. There are many factors, I gave examples. You would need to do some benchmarking of models on target hardware, given certain data. In a very simplistic approach, the number of parameters is a proxy for number of operations, which is why I chose to include that table and also highlight correlations in memory consumption and model accuracy. $\endgroup$
    – n1k31t4
    Commented Feb 8, 2019 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.