I have noticed that more often when training huge Deep Learning models on consumer GPUs (like GTX 1050ti) The network often doesn't work.
The reason is that the GPU just doesn't have enough memory to train the said network. This problem has solutions like using the CPU's cache memory for storage of things that are not being actively being used by GPU.
So my question is - Is there any way to train models on CUDA with memory drawn from System RAM? I understand that the tradeoff would involve speed and accuracy but it JUST might work to train a BERT model on my 1050ti