1
$\begingroup$

I am using Keras=2.3.1 with Tensorflow-gpu=2.0.0 backend. While I trained model on two RTX 2080 ti 11G GPUs, it allocates all data to '/gpu:0',and nothing changed with '/gpu:1'. Surely, the second GPU not used at all.

However, every GPU could work if I selected only one GPU.

Moreover, the two gpus can be run parallelly in Pytorch.

Follow some instances, I try to run multi-gpus with these codes:

enter image description here

enter image description here

Below is NVIDIA-SMI output when I run a multi-gpus model.

enter image description here

and cuda = 10.1, cudnn = 7.6.5.

$\endgroup$
0
$\begingroup$

Check out the docs on TensorFlow GPU usage

If you wanted data parallelism where you run a copy of your model on multiple GPUs and split the data between them, you could use the tf.distribute.MirroredStrategy.

The tf.distribute.Strategy docs are also a good source to read.

Also, you should also profile your application; adding a second GPU has the potential to reduce performance depending on what your bottlenecks are.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.