# Why is my prediction not working?

I have made a TensorFlow model and got the training part working. The model trains successfully and the prediction while training is accurate. Unfortunately, when using the model to only predict, the answer is mostly gibberish and sometimes even blank.

This is the code:

def train_model():
loss_track = []
max_batches = 10001
batches_in_epoch = 1000
try:
for batch in range(max_batches):
fd = next_feed() # This is a wrapper that prepares the correct response (decoder_targets) and returns batch_method(query_list) (batch method returns encoder_inputs, encoder_input_lenghts)
_, l_t_a = sess.run([train_op, loss], fd)
loss_track.append(l_t_a)

if batch == 0 or batch % batches_in_epoch == 0:
#fd = random.choice(list(fd))
#print(fd)
print('batch {}'.format(batch))
print('     minibatch loss: {}'.format(sess.run(loss, fd)))
predict_ = sess.run(decoder_prediction, fd)

for i, (inp, pred) in enumerate(zip(fd[encoder_inputs].T, predict_.T)):
index = random.randint(0, len(fd[encoder_inputs].T)-1)
print('     sample {}'.format(i + 1))
print('     Query                  > {}'.format(batch_to_words(fd[encoder_inputs].T[index])))
print('     Predicted Response     > {}'.format(batch_to_words(predict_.T[index])))
if i >= 2:
break
print()
saver.save(sess, MODEL_LOCATION)
except KeyboardInterrupt:
print('Training interrupted.')

def test_model(sentence):
saver.restore(sess, MODEL_LOCATION)
batch__ = []
query = get_formatted_sentence(sentence) # This returns the array of the sentence. ex. ['hello','world']
batch__.append(
words_to_batch(query) # This is the inverse of batch_to_words(), it transforms the sentence array in an int array, based on the word vocabulary position
)
batch, batch_len = batch_method(batch__)
prediction = sess.run(decoder_prediction, feed_dict={encoder_inputs: batch, encoder_inputs_length: batch_len})
batch_response = []
for i in prediction:
batch_response.append(i[0])
print(batch_to_words(batch_response))


What is wrong here? For the prediction part I insert the query inside another list as batch_method() requires a list.

Keep in mind that for some reason the prediction in the training looks like this: [12,5,3,5,1] (and that is why I am able to call batch_to_words() directly), while in the prediction only mode it returns something like this: [[3],[4],[64],[23],[1],[0],[1]] (Which is very weird)

I also call train_model() or test_model() separately, never both on the same run (that is why I save the model -- as always, I am not sure if I am doing it correctly).

I am not sure at this point if this is to be expected or if I messed up somewhere.

Any help is appreciated.

• Saving and loading looks correct. You could check train loss to see if you are converging and val loss to see if you're overfitting. If both look good, you could run a query from the train set to see if it is "reasonable". If everything looks fine, the last thing to check might be the loss itself. – kenny Jun 11 '18 at 23:37
• The prediction while training (using the exact same code as the prediction-only method) works correctly. I am also starting to think that my model is all broken as in the predictions the <eos> is missing (end of sentence). – A. Dandelion Jun 12 '18 at 11:08