3
$\begingroup$

I am currently studying Deep Learning based Machine Translation systems but I'm not sure in my understanding the logic of the process. I understand that the source and destination language translation sentence pairs must be represented as word2vec vectors, but why is it necessary to apply two (encoder-decoder) Recurrent Neural Networks? My first idea would be applying only one RNN, where the input is the source language examples (in the form of word2vec vectors) and the output is simply a word2vec sequence of the destination language. Why is it necessary to use another RNN?

My additinal question is if this system is flexible enough to cope with synonimes, word order variations and other disambiguities? Is it capable of approximate the correct meaning of a new to-be-translated source language sentence?

And last but not least: how could one evaluate such a model where many translations can be correct at the same time?

$\endgroup$
3
$\begingroup$

In encoder-decoder architecture, we first represent the input sequence by a fixed vector. It is assumed that this fix vector represents the complete meaning of the sentence. Now decoder uses this fix vector to generate the output sequence.

Answers of your questions:

  1. If you use 1 recurrent unit, it outputs the value at each time instant which is not suitable for machine translation as translation of first word may be in the last or somewhere else in target language.(I am not sure how will you map the output and input in that case.)
  2. Regarding synonyms, system can handle synonyms as words which are synonyms have very close word vectors.
  3. Regarding Word-Order, it can learn the word order if you have enough training data. Decoder also uses beam search to find the best possible translation. I will suggest you to read this paper thoroughly. https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf
  4. Regarding evaluation, there are various metrics which are designed to evaluate the machine translation system. BLEU is one of the most famous one out of all. It uses various reference translations. You can read more about BLEU in the paper by Kishore Papineni et. al.(I can not post more than two link in answer as my reputation is less than 10.)

I will also suggest you to read the below paper regarding neural machine translation. https://arxiv.org/pdf/1409.0473.pdf

Hope it helps!!

$\endgroup$
0
$\begingroup$

I might not be understanding you correctly, but to my understanding, to answer your last question, using machine learning (deep learning) for language translation you would ideally implement a system that is scoring the translations. For example, how Google Translate uses deep neural networks is that translates the individual words, and finds the probabilities that they appear next to other words. This is used with training data (gov translations international of UN documents) and a series of possible translations are constructed. A series of all possible translations might I add finding the one that sounds the "most human." How this is relevant to a RNN is that you use the first RNN to add encoding for a sentence. The you use it to decode it. Then you use the parallel corpora data to translate from English to Spanish. Not sure if this answers your question but maybe it helps out a little bit.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.