0
$\begingroup$

When we are concerned about speed, GPU is way better than CPU.

But if I train a model on a GPU and then deploy the same trained model (no quantization techniques used) on a CPU, will this affect the accuracy of my model? Can the accuracy of the same model degrade on a CPU?

My intuition says, GPU vs CPU should not make any difference if accuracy is concerned.

But I have one doubt that, the GPUs and CPUs have their own different ways to process the information internally. Both of them have different architecture. When a model is trained on a GPU, does the exact same way of processing happens when trained on a CPU but in much slower way? I am not concerned about the accuracy while training, but if a model was trained on a GPU, will it perform exactly with same accuracy on a CPU?

$\endgroup$
1
$\begingroup$

The results you get will be identical* for identical inputs. So for any practical purposes, the accuracy you get will not depend on whether you use a cpu or gpu.

*up to floating point precision. From this post on the pytorch forum:

Both are implementing the floating point number computation standard. So they are both correct (even though [they may be] different)

$\endgroup$
1
$\begingroup$

A trained model is just a well-defined set of floats i.e parameters

If we go for prediction,
What it will do are a large number of multiplication and addition of input data and the parameters. This will be done by the ALU (Both CPU and GPU)
A CPU will have a few ALU and a GPU will have thousands of them. So it will complete its task very quickly.

So, unless there is a difference of precision for holding the floats which might accumulate in a Deep Network, I don't see any reason that the prediction should differ.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.