Skip to main content
71 votes
Accepted

How to disable GPU with TensorFlow?

I've seen some suggestions elsewhere, but they are old and do not apply very well to newer TF versions. What worked for me was this: ...
Florin Andrei's user avatar
53 votes
Accepted

Multi GPU in Keras

From the Keras FAQs, below is copy-pasted code to enable 'data parallelism'. I.e. having each of your GPUs process a different subset of your data independently. ...
weiji14's user avatar
  • 656
35 votes
Accepted

Choosing between CPU and GPU for training a neural network

Unlike some of the other answers, I would highly advice against always training on GPUs without any second thought. This is driven by the usage of deep learning methods on images and texts, where the ...
pir's user avatar
  • 790
21 votes
Accepted

Using TensorFlow with Intel GPU

At this moment, the answer is no. Tensorflow uses CUDA which means only NVIDIA GPUs are supported. For OpenCL support, you can track the progress here. BTW, Intel/AMD CPUs are supported. The default ...
Icyblade's user avatar
  • 4,336
19 votes
Accepted

Should I use GPU or CPU for inference?

It is true that for training a lot of the parallalization can be exploited by the GPU's, resulting in much faster training. For Inference, this parallalization can be way less, however CNN's will ...
Laurens Meeus's user avatar
17 votes

Using TensorFlow with Intel GPU

You might want to check out https://github.com/benoitsteiner/tensorflow-opencl/ which is a fork of Tensorflow with OpenCL support. If your OS is supported by the fork and you are able to properly ...
dataoverfit's user avatar
16 votes

How to disable GPU with TensorFlow?

For TF2: ...
tttzof351's user avatar
  • 261
13 votes
Accepted

How to make my Neural Netwok run on GPU instead of CPU

First some stupid sanity-check questions: do you have a GPU in your local machine? (you didn't mention that explicitly). I ask because it will not work e.g. on an integrated Intel graphics card found ...
n1k31t4's user avatar
  • 14.9k
13 votes
Accepted

What are your thoughts on SKLearn's dismissal of GPUs for machine learning?

GPU doesn't inherently fit naturally into all machine learning algorithms. A natural contender is one that inherently takes a myriad of matrix multiplication. This makes sense since graphic ...
Tophat's user avatar
  • 2,430
12 votes

FP16, FP32 - what is it all about? or is it just Bitsize for Float-Values (Python)

FP32 and FP16 mean 32-bit floating point and 16-bit floating point. GPUs originally focused on FP32 because these are the calculations needed for 3D games. Nowadays a lot of GPUs have native support ...
keiv.fly's user avatar
  • 1,269
9 votes
Accepted

After the training phase, is it better to run neural networks on a GPU or CPU?

This depends on many factors, such as the neural network architecture (CNNs tend to be better optimized than RNN on GPU) as well as how many test samples you give as input to the neural network (GPUs ...
Franck Dernoncourt's user avatar
9 votes

CNN memory consumption

I will assume by C1, C2, etc, you mean convolutional layers, and by P1 ,...
StatsSorceress's user avatar
9 votes
Accepted

What is the difference between Pytorch's DataParallel and DistributedDataParallel?

As the Distributed GPUs functionality is only a couple of days old [in the v2.0 release version of Pytorch], there is still no documentation regarding that. So, I had to go through the source code's ...
Dawny33's user avatar
  • 8,296
8 votes
Accepted

Estimating Titan X graphics card impact on performance

There are a lot of parameters which matter when using GPU's for machine learning, some of them are: CUDA core count Memory bandwidth (GB/s) Memory per core (MB) Raw Speed (MHz) Total Memory available ...
Sandeep S. Sandhu's user avatar
8 votes

Should I use GPU or CPU for inference?

Running inference on a GPU instead of CPU will give you close to the same speedup as it does on training, less a little to memory overhead. However, as you said, the application runs okay on CPU. If ...
mpotma's user avatar
  • 366
7 votes

What is the best hardware/GPU for deep learning?

I would recommend you read this article carefully: http://timdettmers.com/2017/04/09/which-gpu-for-deep-learning/ Basically, The most important feature for judging deep learning performance is ...
derekhh's user avatar
  • 211
7 votes
Accepted

What size language model can you train on a GPU with x GB of memory?

Tldr; I’ve seen a good rule-of-thumb is about 14-18x times the model size for memory limits, so for a 10GB card, training your model would max out memory at roughly 540M parameters. There is some ...
brewmaster321's user avatar
6 votes

Switching Keras backend Tensorflow to GPU

I suggest reinstalling the GPU version of Tensorflow, although you can install both version of Tensorflow via virtualenv. GPU version of Tensorflow supports CPU computation, you can switch to CPU ...
Icyblade's user avatar
  • 4,336
6 votes

What is the difference between Pytorch's DataParallel and DistributedDataParallel?

DataParallel is easier to debug, because your training script is contained in one process. DataParallel may also cause poor GPU-...
Alex Yin's user avatar
6 votes
Accepted

GTX 1660 Ti vs. RTX 2060 for a deep learning pc

I would have gone with the RTX. If I was to design deep learning models,I would be prefer more number of cores in the GPU , where RTX is the winner. Lets not forget the Tensor cores which are present ...
Savinay_'s user avatar
  • 139
6 votes

How to disable GPU with TensorFlow?

I find setting the variable outside the script easiest and something that always works. export CUDA_VISIBLE_DEVICES='' Run this on the command line before running ...
momo's user avatar
  • 163
5 votes

Using TensorFlow with Intel GPU

Keras is an abstraction layer for tensorflow/ theano. You need a nvidia card but tensorflow as well as theano can be used with CPU support only. Instructions can be found on their websites.
znarf's user avatar
  • 170
5 votes

Why doesn't training RNNs use 100% of the GPU?

I get about this same utilization rate when I train models using Tensorflow. The reason is pretty clear in my case, I'm manually choosing a random batch of samples and calling the optimization for ...
davidparks21's user avatar
5 votes

What is the best hardware/GPU for deep learning?

Look at the CUDA compute capability. They are a mixture of hardware and software features a GPU has (see guide). I benchmarked the GTX 1070, Titan Black, GTX 970, GTX 980, GTX 980Ti. The numbers can ...
Martin Thoma's user avatar
5 votes

Choosing between CPU and GPU for training a neural network

The CPU is the manager of the branch, he can do a bit of everything, but he is not great at much except delegating tasks. However, the GPU is a dedicated mathematician hiding in your machine. If you ...
JahKnows's user avatar
  • 8,946
5 votes

Should I use GPU or CPU for inference?

You'd only use GPU for training because deep learning requires massive calculation to arrive at an optimal solution. However, you don't need GPU machines for deployment. Let's take Apple's new iPhone ...
SmallChess's user avatar
  • 3,560
5 votes

Multi GPU in Keras

For TensorFlow: TensorFlow Using GPUs Here is the sample code on how is used, so for each task is specified the list with devices/device: ...
n1tk's user avatar
  • 619
5 votes
Accepted

Why do I get an OOM error although my model is not that large?

Why I am getting OOM error on the large batch size although my dataset and model are not that big? Yes, the batch size is probably the reason. Also, another the reason is that you don't use the ...
Antonio Jurić's user avatar
5 votes

Is the PNY NVIDIA Quadro RTX 4000 a good GPU for Machine Learning on Linux?

You seem to be looking at the latest Quatro 4000, which has the following compute rating: You can find the complete list here for all Nvidia GPUs. While it seems to have an impressive score of 7.5 (...
n1k31t4's user avatar
  • 14.9k
5 votes

Why the my Tensorflow code just use one GPU when I assign more than one

If a TensorFlow operation has both CPU and GPU implementations, TensorFlow will automatically place the operation to run on a GPU device first. If you have more than one GPU, the GPU with the lowest ...
ARAT's user avatar
  • 273

Only top scored, non community-wiki answers of a minimum length are eligible