I am trying to run my code on a supercomputer with 8 gpus. Though, I assign 8 gpus but one of them is just occupied. I read some notes in websites and it seems that Tensorflow automatically use gpu if it is applicable but still I don't know how I can use all the gpus. The code is just a deep network to be trained using model.fit() and then predict the test data using model.predict()
If a TensorFlow operation has both CPU and GPU implementations, TensorFlow will automatically place the operation to run on a GPU device first. If you have more than one GPU, the GPU with the lowest ID will be selected by default.
TensorFlow 2.0 now has the
tf.distribute module to distribute training across multiple GPUs, multiple machines or TPUs. It builds on the concept of "distribution strategies". You can use tf.distribute.MirroredStrategy() as a scope, like
strategy = tf.distribute.MirroredStrategy() with strategy.scope(): #DO WHATEVER YOU WANT TO DO HERE
Please check this Distributed training with TensorFlow guide for implementation details and other strategies.