# How do you know when you are using a multi gpu?

I have 2 gpus on my local machines, but i'm not sure that the model I am training is using both of them (the speed has not changed much).

My code:

def get_model():
base_model = ResNet50(weights='imagenet', input_shape=(image_size,image_size,3), include_top=False)
#base_model.trainable = False

model = models.Sequential()
model.summary()

model = multi_gpu_model(model,gpus=2)

#optimizer = optimizers.SGD(lr=1e-4, decay=1e-6, momentum=0.9, nesterov=True)
optimizer = optimizers.RMSprop(lr=0.0001)
model.compile(loss='categorical_crossentropy', optimizer=optimizer, metrics=['acc'])

return model


I just added the multi_gpu_model setting, but am not sure that this is enough. I checked nvidia-smi every 0.5 seconds, but seems like only one gpu is working. How do I make sure that it uses the full 2 gpus?

If you are using Linux and Nvidia GPUs, you can do the following in a terminal

nvidia-smi


Which will show you some stats about the GPUs available on your system.

You can run it automatically every 2 seconds like this to see how power/memory usage changes during training:

watch -n 2 nvidia-smi


If you see nothing or the nvidia-smi command fails, you likely don't have the correct drivers installed.

If theya re showing, but Keras/Tensorflow is not finding them, have a look at this thread for more checks for Tensorflow backend.

• yes I'm using 2 GPU's but I want to know I'm using 2 GPU parallelly when I'm training the neural network – slowmonk Jul 19 '19 at 22:53
• Watching the stats shown in the output of nvidia-smi will show you that. If both are being heavily utilised at the same time, they are being used in parallel. – n1k31t4 Jul 19 '19 at 22:55