Performance of four GTX 1080 Ti's versus one Tesla V100 for Deep Neural Network training

Question

While this may be slightly off-topic, this question does pertain to data science and machine learning. I want to train a VGG16 model on Imagenet from scratch. For this purpose, I am looking into either buying four to six GTX 1080 Ti's or one Tesla V100. I have a feeling that four GTX 1080 Ti's will perform far better than a single V100, but has there been any practical research into this?

EDIT - Each GTX 1080 Ti has 11.3 teraflops of FP32 performance (or 45.2 teraflops for four 1080 Ti's, or slightly less since the scaling isn't linear), whereas the V100 is 100 teraflops, or more than double the performance of four GTX 1080 Ti's. Anyone have any benchmarks for these two on imagenet performance?

Answer 1

This link has more realistic benchmarks of 1080ti vs Titan V (same as the V100 but less RAM), using pytorch. Its not nearly the best case scenario of 10x speedups, and seems to be a more modest 20% to 50% boost. This is fairly expected, nVidias grandiose performance claims tend to be only applicable to fairly restricted benchmarks that don't scale well to real world usage. You'd probably get a lot more out of 4 1080ti cards than a single V100 unless you REALLY need a large amount of RAM in a single card.