Tag Info

43

From the Keras FAQs: https://keras.io/getting-started/faq/#how-can-i-run-a-keras-model-on-multiple-gpus Below is copy-pasted code to enable 'data parallelism'. I.e. having each of your GPUs process a different subset of your data independently. from keras.utils import multi_gpu_model # Replicates model on 8 GPUs. # This assumes that your machine has 8 ...

32

Unlike some of the other answers, I would highly advice against always training on GPUs without any second thought. This is driven by the usage of deep learning methods on images and texts, where the data is very rich (e.g. a lot of pixels = a lot of variables) and the model similarly has many millions of parameters. For other domains, this might not be the ...

15

As for a complete machine learning package on GPU's, no such package exists. However, there are actually a handful of R packages that can use GPU's. You can see these packages on the CRAN High Performance Computing page. You should note that most of these packages do require you to have a NVIDIA card. Of the packages available, there are three packages ...

14

At this moment, the answer is no. Tensorflow uses CUDA which means only NVIDIA GPUs are supported. For OpenCL support, you can track the progress here. BTW, Intel/AMD CPUs are supported.

14

I've seen some suggestions elsewhere, but they are old and do not apply very well to newer TF versions. What worked for me was this: import os os.environ["CUDA_VISIBLE_DEVICES"] = "-1" When that variable is defined and equal to -1, TF uses the CPU even when a CUDA GPU is available.

12

GPU doesn't inherently fit naturally into all machine learning algorithms. A natural contender is one that inherently takes a myriad of matrix multiplication. This makes sense since graphic processors inherently were design for this. However, for an algorithm like a Random Forest this may not be so important. Also there exist a cost to transfer ...

11

You might want to check out https://github.com/benoitsteiner/tensorflow-opencl/ which is a fork of Tensorflow with OpenCL support. If your OS is supported by the fork and you are able to properly install it in your system then you can run Keras on top of it. Note however that integrated GPUs in general do not offer a lot of calculating power, roughly your ...

10

First some stupid sanity-check questions: do you have a GPU in your local machine? (you didn't mention that explicitly). I ask because it will not work e.g. on an integrated Intel graphics card found in some laptops. Second, you installed Keras and Tensorflow, but did you install the GPU version of Tensorflow? Using Anaconda, this would be done with the ...

9

@Dan @SmallChess, I don't completely agree. It is true that for training a lot of the parallalization can be exploited by the GPU's, resulting in much faster training. For Inference, this parallalization can be way less, however CNN's will still get an advantage from this resulting in faster inference. Now you just have to ask yourself: is faster inference ...

8

This depends on many factors, such as the neural network architecture (CNNs tend to be better optimized than RNN on GPU) as well as how many test samples you give as input to the neural network (GPUs can be even faster when given a batch of samples instead of a single sample). As an example, here is a benchmark comparing CPU with GPU on different CNN-based ...

8

There are a lot of parameters which matter when using GPU's for machine learning, some of them are: CUDA core count Memory bandwidth (GB/s) Memory per core (MB) Raw Speed (MHz) Total Memory available (GB) Performance on 16-bit, 32-bit floating ops/sec Tim Dettmers has an excellent (frequently updated) blog where he's compared different cards, near the end ...

7

I will assume by C1, C2, etc, you mean convolutional layers, and by P1 ,P2 you mean pooling layers, and FC means fully connected layers. We can calculate the memory required for a forward pass like this: One image If you're working with float32 values, then following the link provided above by @Alexandru Burlacu you have: Input: 50x50x3 = 7,500 = 7.5K ...

7

I would recommend you read this article carefully: http://timdettmers.com/2017/04/09/which-gpu-for-deep-learning/ Basically, The most important feature for judging deep learning performance is memory bandwidth. GTX 1080 Ti's memory bandwidth is 484 GB/s and TITAN X is 480 GB/s. Read this Quora answer to understand why memory bandwidth is the most important ...

6

I suggest reinstalling the GPU version of Tensorflow, although you can install both version of Tensorflow via virtualenv. GPU version of Tensorflow supports CPU computation, you can switch to CPU easily: with device('/cpu:0'): # your code here I have been using GPU version of Tensorflow on my Tesla K80 for a few months, it works like a charm. Feel free ...

6

As the Distributed GPUs functionality is only a couple of days old [in the v2.0 release version of Pytorch], there is still no documentation regarding that. So, I had to go through the source code's docstrings for figuring out the difference. So, the docstring of the DistributedDataParallel module is as follows: Implements distributed data parallelism at ...

6

Running inference on a GPU instead of CPU will give you close to the same speedup as it does on training, less a little to memory overhead. However, as you said, the application runs okay on CPU. If you get to the point where inference speed is a bottleneck in the application, upgrading to a GPU will alleviate that bottleneck.

5

I get about this same utilization rate when I train models using Tensorflow. The reason is pretty clear in my case, I'm manually choosing a random batch of samples and calling the optimization for each batch separately. That means that each batch of data is in main memory, it's then copied into GPU memory where the rest of the model is, then forward/back ...

5

Look at the CUDA compute capability. They are a mixture of hardware and software features a GPU has (see guide). I benchmarked the GTX 1070, Titan Black, GTX 970, GTX 980, GTX 980Ti. The numbers can be found in my masters thesis (Table 5.3 and Table 5.16), but the gist is: GTX 1070 is by far the fastest GTX 980 and 980 Ti are pretty much the same, but only ...

5

The CPU is the manager of the branch, he can do a bit of everything, but he is not great at much except delegating tasks. However, the GPU is a dedicated mathematician hiding in your machine. If you are doing any math heavy processes then you should use your GPU. Always. If you are using any popular programming language for machine learning such as python ...

4

I was in the same boat, but now that I have figured it out, I have listed the steps for installing tensorflow 0.9 with cuda toolkit 8.0, cudnn 5.1, bazel 0.3 on Ubuntu 16.04 LTS here: http://abhay.harpale.net/blog/machine-learning/deep-learning/getting-tensorflow-to-work-with-gpu-nvidia-gtx-1080-on-ubuntu-16-04-lts/ Here's the gist Install NVidia Cuda ...

4

Solution to my question: If you run a tensorflow session you can choose if the placement of nodes on the different devices is hard or soft coded. In the specific case of CIFAR10 this means: The Queues be placed on the GPU and that's why the error popped up, but: If you turn on soft placement it works. This means, that even though the placement on the GPU of ...

4

As far as I know, we can't control the random seed by adding np.random.seed when it comes to GPU. In fact, the randomness(non-determinstic) is a behavior of GPU. The reason behind is that cuDNN(and othere CUDA stuffs) uses a non-deterministic algorithm to compute gradients, thus we can't determine anything. For theano backend, you can add deterministic ...

4

I'm wondering if a given number of images per second, say 15000, means that 15000 images can be processed by iteration or for fully learning the network with that amount of images?. Typically they specify somewhere whether they talk about the forward (a.k.a. inference a.k.a. test) time, e.g. from the page you mentioned in your question: Another example ...

4

From my experience setting up GPU processing for R is hard, setting it up on a Windows machine is even harder. Additionally, GPU processing can only be used for very specific types of calculations. If you just want to setup GPU processing for the sake of it, then my answer is quite useless. If you however care about general performance optimization of your ...

4

You'd only use GPU for training because deep learning requires massive calculation to arrive at an optimal solution. However, you don't need GPU machines for deployment. Let's take Apple's new iPhone X as an example. The new iPhone X has an advanced machine learning algorithm for facical detection. Apple employees must have a cluster of machines for ...

4

For TensorFlow: TensorFlow Using GPUs Here is the sample code on how is used, so for each task is specified the list with devices/device: # Creates a graph. c = [] for d in ['/gpu:2', '/gpu:3']: with tf.device(d): a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3]) b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2]) c....

4

I would have gone with the RTX. If I was to design deep learning models,I would be prefer more number of cores in the GPU , where RTX is the winner. Lets not forget the Tensor cores which are present in RTX , which makes it stand out for Deep Learning processing. (Faster Tensor operations). Go with RTX if you can afford it.

3

Unfortunately you can't use GTX 580+Tensorflow. TensorFlow GPU support requires having a GPU card with NVidia Compute Capability >= 3.0, while GTX 580 has only 2.0 compute capability. However, caffe supports CUDA compute capability 2.0. You can have a try. References https://www.tensorflow.org/versions/r0.10/get_started/os_setup#...

3

Besides the criteria listed look for The number of cores The ability to do low precision arithmetic In practice you are probably best off just getting the latest generation NVIDIA card (currently 10xx).

3

I'll first reference some quotes from similar questions: When it comes to matrix operations, you don't think twice, you always opt for GPUs. source ‌ The parallel architecture in a GPU is well adapted for vector and matrix operations. source So if you read through these questions, you'll see that they advise to use GPU regardless of the case; ...

Only top voted, non community-wiki answers of a minimum length are eligible