My understanding is that GPUs are more efficient for running neural nets, but someone recently suggested to me that GPUs are only needed for the training phase. Once trained, it's actually more efficient to run them on CPUs.

Is this true?


This depends on many factors, such as the neural network architecture (CNNs tend to be better optimized than RNN on GPU) as well as how many test samples you give as input to the neural network (GPUs can be even faster when given a batch of samples instead of a single sample).

As an example, here is a benchmark comparing CPU with GPU on different CNN-based architectures. The forward pass is much slower on a CPU in that case:

enter image description here

FYI: Benchmarks based on neural networks libraries to compare the performance between different GPUs

  • $\begingroup$ where do you see something about a cpu? $\endgroup$ Jul 7 '20 at 12:49
  • $\begingroup$ @NicolasGervais last row $\endgroup$ Jul 7 '20 at 15:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.