# How to make predictions of multiple input samples at once in tf 2 with keras

I am quite confused on the output of model.predict

when after training I validate my model on around 6000 samples I use the following pseudo code:

model.fit(...)
predictions = model.predict(val_set)
len(predictions) # == len(val_set)


result: tensor array of shape=(len(tensor_array),14) (one prediction for each input sample)

in production I currently use the following code to ocr image numbers:

model = tf.keras.models.load_model('number_ocr_v2')
def predict(list_images):
global model
print("length:")
print(len(list_images))
#predictions = model.predict(list_images) # <- Same result
predictions = model.predict_on_batch(list_images)
print(len(predictions))
print(predictions)


console Output:

length:
30
1
tf.Tensor([[0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]], shape=(1, 14), dtype=float32)

(one prediction for x input samples)

Apparently the Model delivers one array with only the prediction of the first input tensor.

Where are the rest of the 30 Input Tensors? What is the difference between my training code and my production environment that explains why training code delivers predictions[:] and production code only delivers predictions[0]?

A Time consumption benchmark suggests that all 30 digits are beeing processed. Timing fits with 30 Predictions a ~70us per image

Best regards,

Julian