I am building a simple face detection API with Python, Flask and the MTCNN face detector.
My problem is that the model and API are running really quickly (batch of 100 images takes 0.5-0.7 seconds to process) on my workstation (6 core Intel i5 9400F without GPU - macOS 10.15.1), but it has terrible performance (same batch takes 2-3 seconds) on a production server with dual 8 core Intel Xeon E5-2630 v3 (Ubuntu 18.04).
How is this even possible? Is tensorflow somehow accelerated on macOS (Metal API etc)? Should I use a different server?
What have I tried:
- installed Intel optimized tensorflow
- recompiled tensorflow from source to include support for all CPU instructions
- different servers from different vendors
After more tests, it seems that even my MacBook Pro (2.8 GHz Quad-Core Intel Core i7) is outperforming Tensorflow on a GTX 2080 Ti and Tesla T4 (on Ubuntu 18.04), I really don't understand what is going on here.