I've been getting a strange error while trying to run a working trainer as part of my application that involves pytorch and it's tensor class which is confusing me.
It relates to precision.
In my jupiter notebook I define an array [0.1111,0.2222] for example. When I run torch.tensor(array) it returns a tensor with datatype float32. When I run a sequential layer on this eg. self.linear_relu(newtensor) it works fine in the notebook.
But in the Vs Code environment it is defaulting to float64 everytime !
Except in the debug console. eg.
i = float(32) i 32.0 torch.tensor(i) tensor(32.) t=torch.tensor(i) t.dtype torch.float32 t.storage_type <built-in method storage_type of Tensor object at 0x7fd681386080> t.storage_type() <class 'torch.FloatStorage'> x tensor(0.9957681660899654341179144, dtype=torch.float64) x.storage_type() <class 'torch.DoubleStorage'> x.dtype torch.float64
After getting this error I started setting the default data type manually with.
I look in vscode launch and there is no mention of doubles. I don't cast any of my variable to doubles. But here we are.
Of note I am manually casting ALL of the values be placed into tensors to float eg.
All the math values for all the functions I use in python are also returning float types as are all the values in the classes I'm storing values in minus a few integer constants.
Why is the Tensor defaulting to Double, and why is pytorch nn.Linear bombing when it gets the value ?
Meanwhile the Parameters of the network are defaulting to float32, which is the cause of the issue.