I am using tensorflow-gpu 1.10.0 and keras-gpu 2.2.4 with a Nvidia gtx765M(2GB) GPU, OS is Win8.1-64 bit- 16GB RAM.

I can train a network with 560x560 pix images and batch-size=1, but after training is over when I try to test/predict I get the following error:

ResourceExhaustedError: OOM when allocating tensor with shape[20,16,560,560] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
     [[Node: conv2d_2/convolution = Conv2D[T=DT_FLOAT, data_format="NCHW", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](activation_1/Relu, conv2d_2/kernel/read)]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

I suppose it's a memory issue.

So my question is, is it possible to first use GPU for training and then switching CPU to predict some results in one Jupyter notebook?

Can we free up GPU-memory in windows inside a script?

I found these two topics but I think we should use at the beginning of the script. What I want is switching after training.


Switching Keras backend Tensorflow to GPU

Any help will be appreciated, Thanks


1 Answer 1


For prediction step, are you using the same batch size as training ? batch prediction should bring down memory usage.

  • $\begingroup$ Yes, I use batch_size=1. Actually, that is another problem, GPU has 2GB of ram but I can only feed network with batch_size=1(560x560 pix). I think it should be more than that, isn't it? $\endgroup$
    – John
    Commented Mar 19, 2019 at 14:32
  • $\begingroup$ In that case, save the model to a file and use another script for prediction. That is is batch_size=1 is used for prediction. maybe some code / keras / tf issue is preventing GPU memory from being freed. $\endgroup$ Commented Mar 20, 2019 at 3:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.