3
$\begingroup$

Background

I am building a simple face detection API with Python, Flask and the MTCNN face detector.

My problem is that the model and API are running really quickly (batch of 100 images takes 0.5-0.7 seconds to process) on my workstation (6 core Intel i5 9400F without GPU - macOS 10.15.1), but it has terrible performance (same batch takes 2-3 seconds) on a production server with dual 8 core Intel Xeon E5-2630 v3 (Ubuntu 18.04).

Question

How is this even possible? Is tensorflow somehow accelerated on macOS (Metal API etc)? Should I use a different server?

What have I tried:

  • installed Intel optimized tensorflow
  • recompiled tensorflow from source to include support for all CPU instructions
  • different servers from different vendors

Update

After more tests, it seems that even my MacBook Pro (2.8 GHz Quad-Core Intel Core i7) is outperforming Tensorflow on a GTX 2080 Ti and Tesla T4 (on Ubuntu 18.04), I really don't understand what is going on here.

$\endgroup$
  • $\begingroup$ Are the images all loaded in memory before the process starts? Because if not it's probably an I/O bottleneck. $\endgroup$ – Erwan Jan 28 at 0:20
  • $\begingroup$ @Erwan yes, the images are uploaded through an HTTP API and are never stored on disk. $\endgroup$ – Szőke Péter Jan 28 at 9:03
  • $\begingroup$ Have you tried profiling the code to identify where the bottlenecks are? Which steps or function calls are taking the longest? $\endgroup$ – Brian Spiering Mar 26 at 14:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.