Questions tagged [machine-translation]
Machine translation in a data science context refers to the process of using machine learning techniques to translate input provided in one language into output in another. It includes topics referring to using text/corpus data, neural machine translation (NMT), deep learning, and speech recognition.
79
questions
0
votes
1
answer
16
views
What is meant by averaging inhibits it in the paper 'Attention is All You Need'?
Could anyone explain to me about the sentence below? What is meant by averaging inhibits it?
Multi-head attention allows the model to jointly attend to information
from different representation ...
0
votes
0
answers
20
views
Found input variables with inconsistent numbers of samples: [87, 22] [closed]
I'm using neural network machine learning and would like to see the result of my confusion matrix for my model. However, there is an error that I've got and don't know how to solve it
0
votes
0
answers
54
views
How to implement Early stopping in Neural Machine Translation with Attention or Transformers?
I am trying to implement early stopping to my model where I am performing Machine Translation using Seq2Seq with attention. I am mostly used to writing my own models in steps, something like this:
<...
0
votes
1
answer
49
views
WMT: What are the differences of WMT14, WMT15 and WMT16 datasets?
Each year, the Workshop on Statistical Machine Translation (WMT) holds a conference that focuses on new tasks, papers, and findings in the field of machine translation.
Let's say we are talking about ...
- 151
0
votes
1
answer
28
views
For an LSTM-based seq2seq model, is reversing the input still necessary or advised when using attention?
The original seq2seq paper reversed the input sequence and cited multiple reasons for doing so. See:
Why does LSTM performs better when the source target is reversed? (Seq2seq)
But when using ...
- 3
0
votes
0
answers
470
views
Pytorch build_vocab_from_iterator giving vocabulary with very few words
I am trying to build a translation model in pytorch. Following this post on pytorch I downloaded the multi30k dataset and spacy models for English and German.
...
- 3
0
votes
0
answers
24
views
Why does Bahdanau Attention Have to be Causal?
Using the Bahdanau attention layer on Tensorflow for time series prediction, although conceptually it is similar to NLP applications.
This is how the minimal example code for a single layer looks like....
- 283
1
vote
1
answer
318
views
Self Attention vs LSTM with Attention for NMT
I am trying to compare the
A: Transformer-based architecture for Neural Machine Translation (NMT) from the Attention is All You Need paper, with
B: an architecture based on Bi-directional LSTM's in ...
- 11
1
vote
1
answer
29
views
Questions of understanding - Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation
I'm currently analysing the paper Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation (Post, Vilar 2018): https://arxiv.org/abs/1804.06609
I have ...
- 151
0
votes
1
answer
13
views
Training NMT models for noisy social media roman text
I am trying to train an NMT model where the source side is roman text of Asian languages from social media, and target side is English. Note that since roman text is not native to Asia, the ...
- 131
1
vote
2
answers
40
views
Paraphrasing a sentence and changing the tone of it
I am trying to make a model that is capable of translating a sentence into a new and a better form. I would like the model to change the tone and also give it some character. I am using this in my web ...
- 41
3
votes
3
answers
158
views
A good way to organize/store a lot of datasets
In machine translation, we often have bilingual dataset, e.g. for German-English and French-English we will have something that looks like this:
...
- 2,392
1
vote
0
answers
130
views
How to do batch inference on Hugging face pretrained models?
I want to do batch inference on MarianMT model. Here's the code:
...
1
vote
0
answers
24
views
Machine Learning for analyzing and generating sentences from given text inputs
I'm trying to create a program that will translate Sign Language to Text and apply NLP so that the text is understandable to human. I've used CNN for recognizing sign language but I don't know how to ...
0
votes
1
answer
452
views
Reason for adding 1 to word index for sequence modeling
I notice in many of the tutorials 1 is added to the word_index. For example considering a sample code snippet inspired from <...
- 175
0
votes
1
answer
199
views
Why are mBART50 language codes in an unusual format?
I am trying to use mBART for multilingual translation(about 30 languages) but I am facing an issue with using it as I am currently using langid to identify the languages then load mBART and translate ...
- 213
1
vote
1
answer
69
views
Issue translating large amounts of tweets using Google Translate
I am working on translating large amounts of tweets using this deep-translator which uses the Google Translate API.
Initially everything was fine and tweets were translated with no problems whatsoever ...
- 213
-2
votes
1
answer
29
views
I want to start studying the field of machine translation [closed]
I've studied Japanese language and literature and passed some linguistic courses and now as for my masters, I want to study natural language processing and especially machine translation. so I tried ...
0
votes
1
answer
42
views
How to create a Document Categorization Classifier for different contexts of Documents
I have a doubt solving a test. The idea here is to demonstrate the NLP and Machine Translation abilities.
The Dataset is a multilingual, multi-context set of documents. The dataset is divided on ...
- 3
0
votes
1
answer
17
views
Question of pretraining text-generation task, it seems that pretraining is not work for a small model?
My task is to generate keywords from sentences.
I pretrain a text-generation model. I mask the sentences' tokens and predict the whole sentences' tokens.
Pretraining batch_size = 8 and step = 1000000
...
- 561
0
votes
1
answer
32
views
Passing Dependency/Constituency trees to a Neural Machine Translator
I am working on a project on Neural Machine Translation in the English-Irish domain. I am not an expert and have researched entirely on my own for a technology exhibition so apologies if my question ...
2
votes
0
answers
351
views
What is the difference between register_buffer() and parameter.detach() in PyTorch?
I am writing a PositionalEmbedding() module which is an implementation based on "Attention Is All You Need" using PyTorch. According to the paper, there ...
- 21
1
vote
1
answer
221
views
Unusually High BLEU score on a NMT model
This is the project on Neural Machine Translation on the English/Irish language pair. I have been spending the past month or so trying to train a good baseline to do 'experimentation' on. I have a ...
6
votes
2
answers
3k
views
BPE vs WordPiece Tokenization - when to use / which?
What's the general tradeoff between choosing BPE vs WordPiece Tokenization? When is one preferable to the other? Are there any differences in model performance between the two? I'm looking for a ...
- 199
1
vote
1
answer
44
views
Attention network without hidden state?
I was wondering how useful the encoder's hidden state is for an attention network. When I looked into the structure of an attention model, this is what I found a model generally looks like:
...
- 111
0
votes
1
answer
49
views
Predicting correct match of French to English food descriptions
I have a training and test set of food descriptions pairs (please, see example below)
First name in a pair is a name of food in French
and second word is this food description in English.
Traing set ...
- 263
1
vote
0
answers
17
views
Which is the best way to install Moses for SMT?
I have tried many ways but it always ended up in error. My Configuration: Ubuntu 16.04, Python 3.6.10 The error that I am unable to solve is: undefined reference to
...
0
votes
1
answer
312
views
Feeding XLM-R embeddings to neural machine translation?
I’m very new to the field of deep learning. My aim is to make a translation between Catalan to Catalan Sign Language. The grammar of the two languages is different
Input: He sells food.
Output (...
- 31
0
votes
1
answer
635
views
Transformer seq2seq model and loading embeddings from XLM-RoBERTa
Is it possible to feed embeddings from XLM- RoBERTa to transformer seq2seq model? I'm working on NMT that translates verbal language sentences to sign language sentences (e.g Input: He sells food. ...
- 31
1
vote
0
answers
38
views
Seq2Seq for sentence correction
I have a task in hand where I get a dirty formed sentence and need to correct it. Examples are, "StackOverflow best question answering platform" to be converted to "StackOverflow is best question ...
- 794
1
vote
2
answers
621
views
Language translation with convolutional neural network
Many examples of language translation neural networks:
"the cat sat on the mat" -> [model] -> "le chat etait assis sur le tapis"
use RNN, and in particular LSTM. See for example Sentences language ...
- 140
1
vote
0
answers
179
views
How can I finetune XLM-R for neural machine translation between the same language(Catalan to Catalan-with different grammar structure))?
Can I fine-tune XLM-R to do something similar to https://www.guru99.com/seq2seq-model.html
(encoder-decoder architecture or any other way that perform good in small dataset - around 1000)
For example:
...
- 31
2
votes
1
answer
2k
views
Can I fine-tune BERT, ELMO or XLnet for Seq2Seq neural machine translation?
I'm working on neural machine translator that translates English sentences to American sign language sentences(e.g below). I've a quite small dataset - around 1000 sentence pairs. I'm wondering if it ...
- 31
20
votes
2
answers
4k
views
What is the bleu score of professional human translators?
Machine translation models are usually evaluated using bleu score. I want to get some intuition for this score. What is the bleu score of professional human translator?
I know it depends on the ...
- 706
0
votes
1
answer
2k
views
Is it possible feed BERT to seq2seq encoder/decoder NMT (for low resource language)?
I'm working on NMT model which the input and the target sentences are from the same language (but the grammar differs). I'm planning to pre-train and use BERT since I'm working on small dataset and ...
- 31
4
votes
2
answers
177
views
Sentences language translation with neural network, with a simple layer structure (if possible sequential)
Context: Many language sentences translation systems (e.g. French to English) with neural networks use a seq2seq structure:
"the cat sat on the mat" -> [Seq2Seq ...
- 140
9
votes
4
answers
9k
views
Why is the decoder not a part of BERT architecture?
I can't see how BERT makes predictions without using a decoder unit, which was a part of all models before it including transformers and standard RNNs. How are output predictions made in the BERT ...
1
vote
1
answer
78
views
Should the data be shuffled on a translation dataset
As reference to Why should the data be shuffled for machine learning tasks I was curious if this is the case for neural-machine-translation.
I would like to analyze why I think this is the case for ...
- 13
2
votes
1
answer
4k
views
How can I feed BERT to neural machine translation?
I am trying to feed the input and target sentences to an NMT model, I am trying to use BERT here, But I don't have any idea how ...
- 41
1
vote
0
answers
281
views
Weight matrices in transformers
I am trying to understand the transformer architecture.
I am aware that the encoder/decoder contains multiple stacked self attention layers. Further each layer contains multiple heads. For example ...
- 141
1
vote
2
answers
118
views
where to start in natural language processing for a language
My native language is a regional language and few people speak it. I have some assignements in a machine learning course and i was thinking about doing some natural languge processing on my native ...
0
votes
1
answer
455
views
Help making a custom categorical loss function in Keras
I am a bit new to machine learning, and I'm trying to get the basics working towards a bigger project using a very simple encoder-decoder model. It looks like this:
...
- 3
2
votes
0
answers
119
views
Can BERT embeddings be used to reproduce the original content of the text?
From what I understand, BERT provides contextualized embeddings that are not deterministic the way Word2Vec embeddings (i.e. the word "Queen" doesn't always produce the same vector, it'll be different ...
- 21
1
vote
0
answers
18
views
searching for vocabulary for machine translation
Searching for DE-EN vocabulary, that I can use in my machine translation.
It should consist of DE-EN phrases, words, sentences etc.
Got one from here http://www.manythings.org/anki/ , but searching ...
- 21
1
vote
3
answers
1k
views
Using Google Translate API to create a Translation Dataset
Is it a good idea? ;-)
Is it legal to do so? Is it legal to release such a dataset to public?
Say I have a language X for which I want to create a dataset for translation to/from English, for which I ...
- 131
0
votes
1
answer
219
views
NMT, What if we do not pass input for decoder?
For transformer-based neural machine translation (NMT), take English-Chinese for example, we pass English for encoder and use decoder input(Chinese) attend to encoder output, then final output.
What ...
- 561
4
votes
1
answer
964
views
What is Bit Per Character?
What is Bits per Character (bpc) metric which has been used to measure the model accuracy with reference to text8 and ...
- 849
1
vote
1
answer
139
views
Why is hard for neural machine translation model to learn rare words?
i'm kinda new with neural machine translation. I've read some papers, authors usually limit the size of vocabulary by replace rare words by unk token. In this paper, they said that "...NMT model ...
- 131
1
vote
1
answer
61
views
Is it normal when BLEU score on filtered data by length is greater than BLEU score on whole data
I am creating 2 neural machine translation model (model A and B with different improvements each model) with fairseq-py. When I evaluate model with bleu score, model A BLEU score is 25.9 and model C ...
- 131
10
votes
2
answers
2k
views
Multi-Head attention mechanism in transformer and need of feed forward neural network
After reading the paper, Attention is all you need, I have two questions:
1. What is the need of a multi-head attention mechanism?
The paper says that:
"Multi-head attention allows the model to ...
- 233