I searched for other similar questions on Google and here, but couldn't find any. I'm consistently getting results of less quality when making deep learning with GPU, compared to CPU. Of course, it is much faster using GPU, but my results converge to lower quality on many different aspects: subjective quality of embedding, validation accuracy, loss, etc. It doesn't depend on the number of epochs, ie. results won't converge toward the same value: one is always inferior to the other. It's disconcerting when trying to achieve solid, reproducible SOTA results, where .1 or .2% is quite important (litteraly from top2 to top1).
I have a RTX2070 and a good CPU. Is it inherent to modern GPU ? Are approximations done using GPU that could explain this phenomenon ? Thanks for your inputs.