Making predictions / Loading model in TensorFlow 2.0

Question

I use TensorFlow/Keras on a daily basis to make predictions for a project. Everything works fine but I was getting regular warnings about the transition to TensorFlow 2.0 and I thought this week I would finally make sure my code works in the new version of the library as well. I did not encounter any problems during training or saving a model, but when it comes to making predictions I got the following warning:

WARNING:tensorflow:Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least steps_per_epoch * epochs batches (in this case, 10 batches). You may need to use the repeat() function when building your dataset.

Turns out it still does the prediction as expected, but the warning slows down the process considerably. I was able to overcome this by passing steps=1 argument to model.predict(), but this seems like a roundabout way of doing things, which was not needed in the previous version of TensorFlow.

I wonder if I am missing something trivial here. Also, it seems like TensorFlow now fails to figure out that I am doing prediction not training, which also was not an issue before. On a related note, it was probably in the documentation before as well but I never thought about the batch_size argument in model.predict() and the purpose it serves.

Update:

Now that Google Colab changed their default version to TensorFlow 2, I decided the give it another shot. Now, the code is still entirely the same, but there is an error when I try to load a model:

WARNING:tensorflow:Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.

There is an open question on TensorFlow github page on this issue:

https://github.com/tensorflow/tensorflow/issues/37968

I will update again when it is resolved.

Answer 1

I had the same problem in TF 2.1. It has something to do with the shape/ type of the input, namely the query. In my case, I solved the problem as follows: (My model takes 3 inputs)

x = [[test_X[0][0]], [test_X[1][0]], [test_X[2][0]]]
MODEL.predict(x)

Output:

WARNING:tensorflow:Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least steps_per_epoch * epochs batches (in this case, 7 batches). You may need to use the repeat() function when building your dataset.

array([[2.053718]], dtype=float32)

Solution:

x = [np.array([test_X[0][0]]), np.array([test_X[1][0]]), np.array([test_X[2][0]])]
MODEL.predict(x)

Output:

array([[2.053718]], dtype=float32)