I'm building convoluted neural networks with TensorFlow. I've got the latest Mac Pro, use all of my cores, and yet it still takes me several hours to train a single network. I need to do grid search on many different tuning parameters, so this is not ideal.
This is for a personal project, so I want to save money by using the cloud. I have used AWS extensively, and I like the idea of using their spot market, however I've heard that 1) Amazon's GPU instances aren't compatible out of the box with TensorFlow (AWS's GPUs have CUDA 3.0 and TensorFlow requires 3.5) and 2) Amazon's GPUs are a bit slow.
Does Microsoft's Azure or Google's Google Compute Engine have easy-to-use GPUs when using TensorFlow? How do their GPUs compare to those of Amazon in terms of the relevant neural-net-training tech specs, prices, and ease of use? Are there smaller cloud providers that you would recommend instead?
Also, roughly how much faster can I expect training to take on one of these virtual GPUs compared to a top of the line Mac Pro?