0
$\begingroup$

I am facing a memory issue while trying to initialize a torch.zeros - torch.zeros((2000,2000,3200), device=device)

Getting the following error:

RuntimeError: CUDA out of memory. Tried to allocate 47.69 GiB (GPU 0; 8.00 GiB total capacity; 1.50 KiB already allocated; 6.16 GiB free; 2.00 MiB reserved in total by PyTorch)

My question is: why does a zero tensor need that big memory? Or am I doing any mistake?

P.S. I was checking with getsizeof in another system - the size of this tensor is showing as 72bytes only.

$\endgroup$
1
$\begingroup$

The reason the tensor takes up so much memory is because by default the tensor will store the values with the type torch.float32. This data type will use 4kb for each value in the tensor (check using .element_size()), which will give a total of ~48GB after multiplying with the number of zero values in your tensor (4 * 2000 * 2000 * 3200 = 47.68GB). What you could try to do is change the datatype from torch.float32 to something like torch.int8, which only uses 1kb for each value, reducing the memory needed by 75% to 12GB. But given that you only seem to have 8GB available this will also not solve your problem. The only solution therefore would simply be to use a tensor will fewer values.

$\endgroup$
1
  • $\begingroup$ Thanks, I will see if I can reduce the size and will change the float to int. $\endgroup$ Apr 27 at 11:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.