Questions tagged [machine-translation]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
15 views

How can I feed BERT to neural machine translation?

I am trying to feed the input and target sentences to nmt model, I am trying to use BERT ...
0
votes
0answers
13 views

Weight matrices in transformers

I am trying to understand the transformer architecture. I am aware that the encoder/decoder contains multiple stacked self attention layers. Further each layer contains multiple heads. For example ...
1
vote
2answers
23 views

where to start in natural language processing for a language

My native language is a regional language and few people speak it. I have some assignements in a machine learning course and i was thinking about doing some natural languge processing on my native ...
0
votes
1answer
19 views

Help making a custom categorical loss function in Keras

I am a bit new to machine learning, and I'm trying to get the basics working towards a bigger project using a very simple encoder-decoder model. It looks like this: ...
0
votes
0answers
5 views

How to generate alignments for word-based translation models if number of words are different in both sentences

I am working on implementing IBM Model 1. I have a parallel corpus of some 20,00,000 sentences (English to Dutch). Also, the sentences of the two docs are already aligned. Aim is to translate a Dutch ...
1
vote
0answers
13 views

Can BERT embeddings be used to reproduce the original content of the text?

From what I understand, BERT provides contextualized embeddings that are not deterministic the way Word2Vec embeddings (i.e. the word "Queen" doesn't always produce the same vector, it'll be different ...
0
votes
0answers
19 views

nlp: Translation System: Transformer/GPT2 model: Why do we need to mask future tokens?

I am trying to understand the whole concept of masking the tokens in the transformer/gpt2 model. In this blog post, http://jalammar.github.io/illustrated-gpt2/ the author takes an example where " the ...
0
votes
0answers
16 views

searching for vocabulary for machine translation

Searching for DE-EN vocabulary, that I can use in my machine translation. It should consist of DE-EN phrases, words, sentences etc. Got one from here http://www.manythings.org/anki/ , but searching ...
1
vote
3answers
113 views

Using Google Translate API to create a Translation Dataset

Is it a good idea? ;-) Is it legal to do so? Is it legal to release such a dataset to public? Say I have a language X for which I want to create a dataset for translation to/from English, for which I ...
0
votes
1answer
52 views

NMT, What if we do not pass input for decoder?

For transformer-based neural machine translation (NMT), take English-Chinese for example, we pass English for encoder and use decoder input(Chinese) attend to encoder output, then final output. What ...
1
vote
0answers
12 views

What is Bit Per Character?

What is Bits per Character (bpc) metric which has been used to measure the model accuracy with reference to text8 and ...
0
votes
0answers
12 views

What is the impact of <unk> token to quality of machine translation (BLEU)

i've read some papers about machine translation. Authors usually define a threshold to limit vocabulary to minimum rare words and replace rare words into token. So the question is if we increase the ...
1
vote
1answer
26 views

Why is hard for neural machine translation model to learn rare words?

i'm kinda new with neural machine translation. I've read some papers, authors usually limit the size of vocabulary by replace rare words by unk token. In this paper, they said that "...NMT model ...
0
votes
1answer
17 views

Is it normal when BLEU score on filtered data by length is greater than BLEU score on whole data

I am creating 2 neural machine translation model (model A and B with different improvements each model) with fairseq-py. When I evaluate model with bleu score, model A BLEU score is 25.9 and model C ...
0
votes
0answers
31 views

How does embedding layer works in pytorch with neural machine translation?

as i mentioned on title, How does pytorch embedding layer works in machine translation task ? As i know that we can use CBOW or Skip-gram to create pretrained embedding vectors for our translation ...
4
votes
1answer
111 views

Multi-Head attention mechanism in transformer and need of feed forward neural network

After reading the paper, Attention is all you need, I have two questions: 1. What is the need of a multi-head attention mechanism? The paper says that: "Multi-head attention allows the model to ...
0
votes
0answers
31 views

How can I be sure that my neural net is learning?

I have successfully trained a neural net on 100k sentences english->german. The sentences are max. 150 characters long and the characters are normalized to binary like [-1 -1 -1 -1 -1 -1 1] -> A The ...
1
vote
0answers
20 views

Back-Translation model for German and English

Do you know of any pre-trained models for back translation between German and English? I am aware that there are ways to include a monolingual corpus into the training of a machine translation model (...
1
vote
1answer
39 views

Embedding Values in word2vec

Are the embedding values for a particular word using word2vec Skipgram model the weights of the first layer or the softmax output of the function? Does the embedding value change according to the ...
0
votes
0answers
11 views

What will go wrong if we apply linear or other types of regression to translate sentences between two languages?

Disclaimer: I asked the question at https://stats.stackexchange.com/questions/408463/what-will-go-wrong-if-we-apply-linear-or-other-types-of-regression-to-translate, but didn't get any response, so I'...
0
votes
1answer
29 views

What is used for Machine Translation besides RNN

I am doing a university report and it seems that encode-decode RNN are optimal for Machine Translation. I need something else to compare it to but I can't seem to make a proper google search for it. ...
1
vote
1answer
136 views

How to build a machine translation system for a new language

I am trying to figure out what are the options for building a natural language translation model for a language that is not yet supported by existing machine translations. The project is to build a ...
0
votes
0answers
188 views

Implementing back translation as a data augmentation for text classification

Since back translation English->other language -> English seems like quite a useful data augmentation technique , I wanted to experiment with it. E.g. it occurred to me that languages from very ...
0
votes
0answers
30 views

How can I increase CUDA load on a Tensorflow deep learning task after reducing batch size to fit the GPU?

I am running this TensorFlow task for Swahili-to-English on an NVidia GeForce 1060 GPU with 6GB of VRAM: https://colab.research.google.com/github/tensorflow/tensorflow/blob/master/tensorflow/contrib/...
2
votes
1answer
72 views

How to create a language translator from scratch?

I want to create a translator which can translate English, Korean and Tamil sentences into English sentence, I tried googletrans but is there any way to create something better than that using DL and ...
0
votes
0answers
42 views

Is it possible to build a univariate decision tree which classifies exactly similar to a 1-NN using the Euclidean distance measure

The main objective is here 1NN with Euclidean distance model can be similar to the univariate decision tree.
2
votes
0answers
634 views

How to train tensorflow's transformer model on my own data?

https://github.com/tensorflow/models/blob/master/official/transformer has an implementation of transformer model. I want to train the model on my own data(consisting of two files, src.txt, and tgt.txt)...
2
votes
1answer
30 views

Statistical machine translation word alignment for FR-ENG and ENG-FR: what is p(e) and p(f)?

I'm currently trying to implement this paper, but am struggling to understand some of the math here. I'm pretty sure I understand how to implement the E-step, but for the M-step, I'm confused on how ...
2
votes
1answer
167 views

word/sentence alignment for English document

I have a English document, which is preprocessed into two versions. I want to align words or sentences from these two versions of document. A simple example is as below: ...
2
votes
1answer
525 views

Is there a dataset with news articles and their headlines? [closed]

I need a set of news headlines and articles to help me in a project on automatic summarization. Is there such a dataset or something similar?
2
votes
0answers
105 views

Encoder-Decoder Sequence-to-Sequence Model for Translations in Both Directions

Is it possible to use a pre-trained sequence to sequence encoder-decoder model which translates an input text in source language to an output in target language to do an inverse translation? That is, ...
1
vote
0answers
15 views

How can we predict Geo Tagged Pollution Parameters using a Machine Learning Model with an Android Device?

Newly installed Pollution Control Sensory Platforms are dumping Continous/Discrete Environment Pollution Parametric Data from various locations. Whats the best format structured or Unstructured of ...
0
votes
1answer
510 views

Do we really need `<unk>` tokens?

I am wondering, do we really need <unk> tokens? Why do we limit our vocabulary? Is it for speed? Accuracy? If we disable all limitations, what do you ...
4
votes
2answers
5k views

Is there “Attention Is All You Need” implementation in Keras?

Has anyone seen this model's implementation using Keras? inb4: tensorflow, pytorch
3
votes
1answer
4k views

Problem with keras model loading

I want to use this keras seq2seq example to train my model. But i dont undestand role of encoder and decoder model and why we dont use model which we trained here. model.fit([encoder_input_data, ...
1
vote
0answers
106 views

Algorithm to parse PSD into html/XML?

I have been working on a project and we were trying to convert a PSD file to a HTML for web applications as well as a Layout XML for android. We worked our way to generate basic skeletal html/xml but ...
3
votes
1answer
980 views

What is the BLEU score used in Google Brain's “Attention Is All You Need” paper?

Google Brain's Attention Is All You Need paper on sequence-to-sequence translation reports: Our model achieves 28.4 BLEU on the WMT 2014 Englishto-German translation task, improving over the ...
1
vote
0answers
37 views

What does NIST information weights refer to?

NIST is a metric used to measure the goodness of translation. In the paper, Doddington (2002) introduce the notion of "Information weights" Information weights were computed using N-gram counts ...
1
vote
0answers
311 views

Hindi English Machine translation using OpenNMT

I am trying to perform machine translation using https://github.com/OpenNMT/OpenNMT-py/ I am using this dataset for training. https://github.com/karimkhanp/Seq2Seq/tree/master/data Result looks like ...
1
vote
1answer
1k views

Number of parameters in seq2seq model

I am trying to compute the total number of parameters in Ilya Sutskever's seq2seq model. In the paper, the authors mention a total number of 384M trainable parameters, or to be exact: The resulting ...
3
votes
1answer
1k views

Why does LSTM performs better when the source target is reversed? (Seq2seq)

I'm currently reading this paper : https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf Here, the author mentioned that LSTM learns better when the order of the ...
0
votes
1answer
39 views

What's the difference of Symbolic translation approach and statistical translation?

Based on what I know, symbolic computing is based on hard rules in the code. Statistical translation is based on statistic and probabilities. If so, could you give an easy to understand example to ...
3
votes
1answer
2k views

Tensorflow speech-to-text training on single file with tf.nn.ctc_loss not converging

I want to train speech-to-text model on tensorflow. Code at: https://github.com/bikramjitroy/speech-to-text/blob/master/TrainingUnitTest-CTCTrain.py A sample "audio.wav" and "label.txt" file also ...
3
votes
2answers
207 views

High Level Understanding of Neural Machine Translation

I am currently studying Deep Learning based Machine Translation systems but I'm not sure in my understanding the logic of the process. I understand that the source and destination language translation ...
1
vote
1answer
2k views

What is context window size?

I am trying to implement a recurrent neural network machine translation system, and I am just learning the things. I am creating a word embedding matrix. In order to do that, I should know my ...
8
votes
2answers
2k views

What's an LSTM-LM formulation?

I am reading this paper "Sequence to Sequence Learning with Neural Networks" http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf Under "2. The Model" it says: ...
2
votes
0answers
33 views

How to estimate the lambda e to reweighing translation probabilities of entries in a dictionary

In this paper, Brown et al. proposed the usage of dictionary entries in addition to the machine translation probabilities that we're used to in the Math of Statistical Machine Translation. He ...
1
vote
0answers
67 views

Minimum Error Rate Training using Powell Search for Machine Translation

From the tutorial slides: http://mt-class.org/jhu/slides/lecture-tuning.pdf, (slide 37) the powell search algorithm goes as such: ...
4
votes
2answers
235 views

How to get phrase tables from word alignments?

The output of my word alignment file looks as such: ...