I am using tensorflow-gpu 1.10.0 and keras-gpu 2.2.4 with a Nvidia gtx765M(2GB) GPU, OS is Win8.1-64 bit- 16GB RAM.

I can train a network with 560x560 pix images and batch-size=1, but after training is over when I try to test/predict I get the following error:

ResourceExhaustedError: OOM when allocating tensor with shape[20,16,560,560] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
     [[Node: conv2d_2/convolution = Conv2D[T=DT_FLOAT, data_format="NCHW", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](activation_1/Relu, conv2d_2/kernel/read)]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

I suppose it's a memory issue.

So my question is, is it possible to first use GPU for training and then switching CPU to predict some results in one Jupyter notebook?

Can we free up GPU-memory in windows inside a script?

I found these two topics but I think we should use at the beginning of the script. What I want is switching after training.


Switching Keras backend Tensorflow to GPU

Any help will be appreciated, Thanks


For prediction step, are you using the same batch size as training ? batch prediction should bring down memory usage.

| improve this answer | |
  • $\begingroup$ Yes, I use batch_size=1. Actually, that is another problem, GPU has 2GB of ram but I can only feed network with batch_size=1(560x560 pix). I think it should be more than that, isn't it? $\endgroup$ – John Mar 19 '19 at 14:32
  • $\begingroup$ In that case, save the model to a file and use another script for prediction. That is is batch_size=1 is used for prediction. maybe some code / keras / tf issue is preventing GPU memory from being freed. $\endgroup$ – Shamit Verma Mar 20 '19 at 3:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.