6
$\begingroup$

I need to purchase some GPUs, which I plan to use for training and using some neural networks (most likely with Theano and Torch). Which GPU specifications should I pay attention to?

E.g.:

What else matter, and what else doesn't matter?

E.g., is it reasonable to assume that the number of sockets, clock speed and the number of cores do not bring any additional useful information (since we already consider the number of Tflops)?

enter image description here

$\endgroup$
3
$\begingroup$

Besides the criteria listed look for

  • The number of cores
  • The ability to do low precision arithmetic

In practice you are probably best off just getting the latest generation NVIDIA card (currently 10xx).

$\endgroup$
  • $\begingroup$ Thanks. Why would the number of cores matter if we have already looked at the Tflops? Good point regarding the ability to do low precision arithmetic, e.g. Theano 0.8 supports float 16. $\endgroup$ – Franck Dernoncourt May 8 '16 at 18:22
2
$\begingroup$

It depends on the expected size of neural network you are thinking about. If the DNN consists multiple layers with large default input and many fully connected layers then you need a GPU with large memory.

Memory is the most important factor , without enough memory, you GPU is useless. To get an idea about the size of any DNN try to think about the number and size of fully connected layers. Of course the number of cores is important but again memory is more important

$\endgroup$
  • $\begingroup$ Yes, definitely if one doesn't have enough RAM, the GPU won't be very useful. $\endgroup$ – Franck Dernoncourt May 8 '16 at 18:26
1
$\begingroup$

I would add a few more parameters to consider:

  1. Size of cache memory associated with each cuda core.
  2. Video memory throughput (Gb/s), higher is better.

If a particular GPU's memory falls short for your dataset, one alternative is to work on a sample of your training set. Although this may not be possible in some cases.

Another alternative to deal with large datasets is if your're comfortable porting algorithms by writing your own CUDA kernels, then you could overcome memory limitations by breaking up your dataset horizontally and swapping each part in/out till the entire dataset is processed. It is quite cumbersome to implement the backpropagation in this manner, and I haven't tried implementing it in this manner on a GPU, so it would be interesting to get feedback on your progress.

$\endgroup$
1
$\begingroup$

Make sure computing capacity is sufficient for your software frameworks.

i.e. tensorflow 1.11 just dropped CP=3.0

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.