# Trained Tensorflow model performs poorly on inference

I trained an image classification model using Keras with Tensorflow backend. The model got good accuracy on validation dataset as well as on the testing data, I save the entire model to .h5 format, here is my checkpoint callback.

checkpoint = ModelCheckpoint(model_name+".h5", monitor='val_acc', verbose=1, save_best_only=True, save_weights_only=False, mode='auto', period=1).

As I was hoping to use this model on Android so I refreeze the model to binary protobuf (.pb) using keras_to_tensorflow.

When performing inference using the model on mobile I noticed the model gives very wrong and random predictions. I have tried exploring other reasons why this could be the case like I found here still, it seems clear the issue is not with loading the images.

Also, performing inference with the converted model on Tensorflow Python still gives the same wrong/random predictions. Here is my code for performing inference in Python.

def model_predict( model_path, image_path, model_input, model_output, class_names ):

with tf.Graph().as_default() as graph: # Set default graph as graph

with tf.Session() as sess:
# Load the graph in graph_def
print("load graph")

# We load the protobuf file from the disk and parse it to retrive the unserialized graph_drf
with gfile.FastGFile(model_path,'rb') as f:

print("Load Image...")
# Read the image & get statstics
np_image = Image.open(image_path)
np_image = np.array(np_image).astype('float32')/255
np_image = np.resize(np_image, (224, 224, 3))
np_image = np.expand_dims(np_image, axis=0)

# Set FCN graph to the default graph
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
sess.graph.as_default()

# Import a graph_def into the current default Graph (In this case, the weights are (typically) embedded in the graph)

tf.import_graph_def(
graph_def,
input_map=None,
return_elements=None,
name="",
op_dict=None,
producer_op_list=None
)

# INFERENCE Here
m_input = graph.get_tensor_by_name(model_input) # Input Tensor
m_output = graph.get_tensor_by_name(model_output) # Output Tensor

print ("Shape of input : ", tf.shape(m_input))
#initialize_all_variables
tf.global_variables_initializer()

# Run model on single image
Session_out = sess.run( m_output, feed_dict = {m_input : np_image} )

print("Predicted class:", class_names[Session_out[0].argmax()] )


How do I perform inference using Tensorflow Python/Android with a save .pb model?

Others have suggested I save the session used for training and load them to Tensorflow when performing inference. If this is the case how do I load the saved session in Tensorflow android?

I am sure the model did not overfit, it performs very well when using Keras.

## 2 Answers

I believe the samples that you are using for inference belongs to a different distribution of data from training and test set. So, I would suggest you to verify if the above case is true and if it is then you should fine-tune your model for the examples that you are trying to carry out inference.

• Performing inference with the model using Keras gives correct predictions, I only get random predictions when using the model in Tensorflow. – Tobi Obadiah Sep 13 '19 at 8:50

See the model must be overfitting as the prediction code looks correct. Kindly check for other metrices than only using accuracy. Print the confidence matrix and see the results.