I trained an image classification model using Keras with Tensorflow backend. The model got good accuracy on validation dataset as well as on the testing data, I save the entire model to .h5 format, here is my checkpoint callback.

checkpoint = ModelCheckpoint(model_name+".h5", monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=False, mode='auto', period=1).

As I was hoping to use this model on Android so I refreeze the model to binary protobuf (.pb) using keras_to_tensorflow.

When performing inference using the model on mobile I noticed the model gives very wrong and random predictions. I have tried exploring other reasons why this could be the case like I found here still, it seems clear the issue is not with loading the images.

Also, performing inference with the converted model on Tensorflow Python still gives the same wrong/random predictions. Here is my code for performing inference in Python.

def model_predict( model_path, image_path, model_input, model_output, class_names ):

    with tf.Graph().as_default() as graph: # Set default graph as graph

        with tf.Session() as sess:
            # Load the graph in graph_def
            print("load graph")

            # We load the protobuf file from the disk and parse it to retrive the unserialized graph_drf
            with gfile.FastGFile(model_path,'rb') as f:

                print("Load Image...")
                # Read the image & get statstics
                np_image = Image.open(image_path)
                np_image = np.array(np_image).astype('float32')/255
                np_image = np.resize(np_image, (224, 224, 3))
                np_image = np.expand_dims(np_image, axis=0)

                # Set FCN graph to the default graph
                graph_def = tf.GraphDef()

                # Import a graph_def into the current default Graph (In this case, the weights are (typically) embedded in the graph)


                # INFERENCE Here
                m_input = graph.get_tensor_by_name(model_input) # Input Tensor
                m_output = graph.get_tensor_by_name(model_output) # Output Tensor

                print ("Shape of input : ", tf.shape(m_input))

                # Run model on single image
                Session_out = sess.run( m_output, feed_dict = {m_input : np_image} )

                print("Predicted class:", class_names[Session_out[0].argmax()] )

How do I perform inference using Tensorflow Python/Android with a save .pb model?

Others have suggested I save the session used for training and load them to Tensorflow when performing inference. If this is the case how do I load the saved session in Tensorflow android?

I am sure the model did not overfit, it performs very well when using Keras.


2 Answers 2


I believe the samples that you are using for inference belongs to a different distribution of data from training and test set. So, I would suggest you to verify if the above case is true and if it is then you should fine-tune your model for the examples that you are trying to carry out inference.

  • $\begingroup$ Performing inference with the model using Keras gives correct predictions, I only get random predictions when using the model in Tensorflow. $\endgroup$ Sep 13, 2019 at 8:50

See the model must be overfitting as the prediction code looks correct. Kindly check for other metrices than only using accuracy. Print the confidence matrix and see the results.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.