# Using CPU after training in GPU

I am using tensorflow-gpu 1.10.0 and keras-gpu 2.2.4 with a Nvidia gtx765M(2GB) GPU, OS is Win8.1-64 bit- 16GB RAM.

I can train a network with 560x560 pix images and batch-size=1, but after training is over when I try to test/predict I get the following error:

ResourceExhaustedError: OOM when allocating tensor with shape[20,16,560,560] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[Node: conv2d_2/convolution = Conv2D[T=DT_FLOAT, data_format="NCHW", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](activation_1/Relu, conv2d_2/kernel/read)]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.


I suppose it's a memory issue.

So my question is, is it possible to first use GPU for training and then switching CPU to predict some results in one Jupyter notebook?

Can we free up GPU-memory in windows inside a script?

I found these two topics but I think we should use at the beginning of the script. What I want is switching after training.

https://github.com/keras-team/keras/issues/4613

Switching Keras backend Tensorflow to GPU

Any help will be appreciated, Thanks

For prediction step, are you using the same batch size as training ? batch prediction should bring down memory usage.

• Yes, I use batch_size=1. Actually, that is another problem, GPU has 2GB of ram but I can only feed network with batch_size=1(560x560 pix). I think it should be more than that, isn't it? – John Mar 19 '19 at 14:32
• In that case, save the model to a file and use another script for prediction. That is is batch_size=1 is used for prediction. maybe some code / keras / tf issue is preventing GPU memory from being freed. – Shamit Verma Mar 20 '19 at 3:58