0
$\begingroup$

I've been getting a strange error while trying to run a working trainer as part of my application that involves pytorch and it's tensor class which is confusing me.

It relates to precision.

In my jupiter notebook I define an array [0.1111,0.2222] for example. When I run torch.tensor(array) it returns a tensor with datatype float32. When I run a sequential layer on this eg. self.linear_relu(newtensor) it works fine in the notebook.

But in the Vs Code environment it is defaulting to float64 everytime !

Except in the debug console. eg.

i = float(32)
i
32.0
torch.tensor(i)
tensor(32.)
t=torch.tensor(i)
t.dtype
torch.float32
t.storage_type
<built-in method storage_type of Tensor object at 0x7fd681386080>
t.storage_type()
<class 'torch.FloatStorage'>
x[0]
tensor(0.9957681660899654341179144, dtype=torch.float64)
x[0].storage_type()
<class 'torch.DoubleStorage'>
x[0].dtype
torch.float64

After getting this error I started setting the default data type manually with.

torch.set_default_dtype(torch.float32)

I look in vscode launch and there is no mention of doubles. I don't cast any of my variable to doubles. But here we are.

Of note I am manually casting ALL of the values be placed into tensors to float eg.

float(value)

All the math values for all the functions I use in python are also returning float types as are all the values in the classes I'm storing values in minus a few integer constants.

Why is the Tensor defaulting to Double, and why is pytorch nn.Linear bombing when it gets the value ?

Meanwhile the Parameters of the network are defaulting to float32, which is the cause of the issue.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.