0
$\begingroup$

I am trying to run some PyTorch scripts on a remote GPU server. While calling the script in the ubuntu terminal i start as:CUDA_VISIBLE_DEVICES=0(or whichever is available) python3 <script.py>. Also, used the following snippet in the code and used .to(device) on the model, input and target tensors.

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(device)

I have confirmed that my model and data and target tensors are mounted on the cuda device. But the GPU shows 0 percentage utilization all through the run. What could I be missing?

$\endgroup$
2
  • $\begingroup$ How are you monitoring your GPU utilization? On windows, I tend to use nvidia-smi. $\endgroup$ Commented Apr 5, 2021 at 15:59
  • $\begingroup$ Yeah, same for ubuntu. $\endgroup$ Commented Apr 6, 2021 at 17:30

1 Answer 1

0
$\begingroup$

The problem was due to some modules I was importing without ensuring the tensors are mounted on cuda through that part of the pipeline. This had turned into the bottleneck blocking the GPU utilization.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.