If I call model.cuda() in pytorch where model is a subclass of nn.Module, and say if I have four GPUs, how it will utilize the four GPUs and how do I know which GPUs that are using?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Please consider marking the answer as correct, or alternatively commenting why you think it is not correct. $\endgroup$– noeFeb 16, 2021 at 22:49
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
model.cuda() by default will send your model to the "current device", which can be set with torch.cuda.set_device(device).
An alternative way to send the model to a specific device is model.to(torch.device('cuda:0')).
This, of course, is subject to the device visibility specified in the environment variable CUDA_VISIBLE_DEVICES.
You can check GPU usage with nvidia-smi. Also, nvtop is very nice for this.
The standard way in PyTorch to train a model in multiple GPUs is to use nn.DataParallel which copies the model to the GPUs and during training splits the batch among them and combines the individual outputs.
-
$\begingroup$ If a user has two GPU's, and the 1st GPU is AMD which is not cuda-capable, and the 2nd GPU is NVIDIA, will it still be correct to say model.half().to("cuda:0"), or do I need some logic to check the index (i.e. would the correct input be "cuda:1" in that case and if so how can I detect that?) $\endgroup$– peteApr 5, 2022 at 9:22
-
-
$\begingroup$ Ok, I asked here datascience.stackexchange.com/questions/109682/… $\endgroup$– peteApr 5, 2022 at 10:56