I am trying to perform Hindi To English translation using s2s model, following - https://chunml.github.io/ChunML.github.io/project/Sequence-To-Sequence/
I have used https://github.com/karimkhanp/Seq2Seq/tree/master/data dataset which contains 37726 sentences for training.
Training took around 44 hours on my 8GB machine. I considered 3 layers, 10 epoch, 20000 vocap size
Other parameters
ap.add_argument('-max_len', type=int, default=200)
ap.add_argument('-vocab_size', type=int, default=20000)
ap.add_argument('-batch_size', type=int, default=100)
ap.add_argument('-layer_num', type=int, default=3)
ap.add_argument('-hidden_dim', type=int, default=1000)
ap.add_argument('-nb_epoch', type=int, default=10)
ap.add_argument('-mode', default='train')
But now when I test using hindi sentence, it give UNK for all words. Though I take same sentence for testing as in training even it says UNK in results.
Test sentences:
डेली हिन्दी न्यूज - बुंदेलखंड का प्रथम अन्तरजालीय स्थल
मैं भारत से प्यार करता हूँ
ज़बूर जो कि दाउद को प्रदान की गयी
प्रशासनिक विभाजन
वे इस देश के प्रथम UNK -LRB- अफ्रीकी UNK -RRB-
नेपाली विदेश
वेल्श खिलाड़ी इंग्लैंड के लिए खेलने के लिए पात्र हैं
उनके बड़े भाई अजीत तेंडुलकर ने उन्हें खेलने के लिये प्रोत्साहित किया था ।
फिर एक मिनट के बाद किताब छत की ओर उछालकर उन्होंने कहा - चलो ।
Result :
of of of
of of of
the of of of
lrb lrb rrb rrb
the the of of of
the the of of of of
the the the of of of of
I could not understand the issue behind this result. I have used some sentences which are same as training sentences. Atleast answer should be correct for them.
Did I do anything wrong? I really appreciate any help
Answer -
if MODE == 'train':
k_start = 1
t1 = time()
# If any trained weight was found, then load them into the model
if len(saved_weights) != 0:
print('[INFO] Saved weights found, loading...')
epoch = saved_weights[saved_weights.rfind('_')+1:saved_weights.rfind('.')]
model.load_weights(saved_weights)
k_start = int(epoch) + 1
i_end = 0
for k in range(k_start, NB_EPOCH+1):
# Shuffling the training data every epoch to avoid local minima
indices = np.arange(len(X))
np.random.shuffle(indices)
X = X[indices]
y = y[indices]
# Training 1000 sequences at a time
for i in range(0, len(X), 1000):
if i + 1000 >= len(X):
i_end = len(X)
else:
i_end = i + 1000
y_sequences = process_data(y[i:i_end], y_max_len, y_word_to_ix)
print('[INFO] Training model: epoch {}th {}/{} samples'.format(k, i, len(X)))
model.fit(X[i:i_end], y_sequences, batch_size=BATCH_SIZE, nb_epoch=1, verbose=2)
model.save_weights('checkpoint_epoch_{}.hdf5'.format(k))
print("Time taken to train the data in hour=>", (time()-t1)/3600)
# Performing test if we chose test mode
else:
# Only performing test if there is any saved weights
if len(saved_weights) == 0:
print("The network hasn't been trained! Program will exit...")
sys.exit()
else:
# import pdb
# pdb.set_trace()
X_test = load_test_data('test', X_word_to_ix, MAX_LEN)
X_test = pad_sequences(X_test, maxlen=X_max_len, dtype='int32')
model.load_weights(saved_weights)
# print(X_test)
# print(model.predict(X_test))
predictions = np.argmax(model.predict(X_test), axis=2)
# print(predictions)
# print(y_ix_to_word)
sequences = []
for prediction in predictions:
sequence = ' '.join([y_ix_to_word[index] for index in prediction if index > 0])
print(sequence)
sequences.append(sequence)
