I am trying to run my code on a supercomputer with 8 gpus. Though, I assign 8 gpus but one of them is just occupied. I read some notes in websites and it seems that Tensorflow automatically use gpu if it is applicable but still I don't know how I can use all the gpus. The code is just a deep network to be trained using model.fit() and then predict the test data using model.predict()


1 Answer 1


If a TensorFlow operation has both CPU and GPU implementations, TensorFlow will automatically place the operation to run on a GPU device first. If you have more than one GPU, the GPU with the lowest ID will be selected by default.

TensorFlow 2.0 now has the tf.distribute module to distribute training across multiple GPUs, multiple machines or TPUs. It builds on the concept of "distribution strategies". You can use tf.distribute.MirroredStrategy() as a scope, like

strategy = tf.distribute.MirroredStrategy()
with strategy.scope():

Please check this Distributed training with TensorFlow guide for implementation details and other strategies.

  • $\begingroup$ Thanks, that worked. Now I have issues with cudnn!! $\endgroup$ Mar 6, 2021 at 23:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.