Questions tagged [machine-translation]
Machine translation in a data science context refers to the process of using machine learning techniques to translate input provided in one language into output in another. It includes topics referring to using text/corpus data, neural machine translation (NMT), deep learning, and speech recognition.
89
questions
1
vote
1
answer
46
views
Unsupervised Machine Translation System Using Variational Autoencoder Models
I want to work on an unsupervised machine translation system using a variational autoencoder. I did a literature review but didn't find any related work, and most of the work is based on denoising ...
0
votes
2
answers
50
views
Why are there no automated translated subtitles?
This might be a rather naive question. There are programs that can automate subtitles, and also programs that can automate translations. Why, then, is there apparently no program that can automate ...
- 265
0
votes
0
answers
7
views
Systematic way of selecting internet texts for a machine translation corpus / dataset?
I am currently working on a neural machine translation project and want to gather a corpus (or dataset) of internet texts that are written in standard and plain language. In theory, it certainly makes ...
- 101
0
votes
1
answer
23
views
How to evaluate machine translations of long documents?
I'm using Python and I want to compare the output of two machine-translation (ish) systems. Most of the tools seem to be focused on sentence-by-sentence evaluation. Either I get memory blow-ups with ...
- 101
0
votes
0
answers
24
views
Extremely silly transformers fine-tuning doubt
Suppose say I have a TFT5ConditionalGenration model instantiated with tensor flow.
When you do model.fit, the model will pass the inputs in a feed-forward way similar to a call method. The model will ...
0
votes
0
answers
493
views
How to Fine-Tune Multilingual Translational Transformer Models
I'm trying to fine-tune facebook/nllb-200-distteled-600M model on my english to arabic dataset, but the results are so bad I'm expecting there should be something wrong with defining the model and ...
0
votes
0
answers
132
views
How to fine-tune a large language model for translation in a multi-dataset setting?
The Problem
We need to translate from language N to language C. If it helps, N is a natural ...
2
votes
0
answers
46
views
How much data and computation power do I need to train a machine translation model using Transformer architecture?
I am working right now on creating a dataset to use in creating a machine translation model to translate between two dialects. I have two questions that I am trying to find an answer for:
How much ...
- 21
0
votes
1
answer
565
views
How to translate text automatically using Google Translate API (or any other approach) in python
I have a dataset of reviews from TripAdvisor and I would like to translate non-English reviews into English.
The reviews are in many different languages: my dataset contains reviews in 44 different ...
0
votes
0
answers
24
views
Is it possible for computers to tell how many people are speaking in a audio recording
Suppose I had a audio recording of 15 students saying "Here" all at the same. Can I tell how many students were speaking and who they were using machine learning?
I want to create a school ...
- 101
0
votes
1
answer
173
views
Extract the embedding from a specific layer of MarianModel
I am using using MarianModel from the hub of HuggingFace for a translation task. Now I want to extract the embedding from the output of the last MarianEncoderLayer ...
- 231
1
vote
1
answer
65
views
What could cause pre-trained Opus-MT models have wildly varying inference time when being used with transformers library?
I have been testing pre-trained Opus-MT models ported to transformers library for python implementation. Specifically, I am using opus-mt-en-fr for English to French translation. And the tokenizer and ...
- 131
0
votes
2
answers
102
views
What is meant by averaging inhibits it in the paper 'Attention is All You Need'?
Could anyone explain to me about the sentence below? What is meant by averaging inhibits it?
Multi-head attention allows the model to jointly attend to information
from different representation ...
- 1
0
votes
1
answer
278
views
WMT: What are the differences of WMT14, WMT15 and WMT16 datasets?
Each year, the Workshop on Statistical Machine Translation (WMT) holds a conference that focuses on new tasks, papers, and findings in the field of machine translation.
Let's say we are talking about ...
- 151
0
votes
1
answer
159
views
For an LSTM-based seq2seq model, is reversing the input still necessary or advised when using attention?
The original seq2seq paper reversed the input sequence and cited multiple reasons for doing so. See:
Why does LSTM performs better when the source target is reversed? (Seq2seq)
But when using ...
- 3
0
votes
0
answers
2k
views
Pytorch build_vocab_from_iterator giving vocabulary with very few words
I am trying to build a translation model in pytorch. Following this post on pytorch I downloaded the multi30k dataset and spacy models for English and German.
...
- 3
1
vote
0
answers
82
views
Why does Bahdanau Attention Have to be Causal?
Using the Bahdanau attention layer on Tensorflow for time series prediction, although conceptually it is similar to NLP applications.
This is how the minimal example code for a single layer looks like....
- 315
1
vote
2
answers
2k
views
Self Attention vs LSTM with Attention for NMT
I am trying to compare the
A: Transformer-based architecture for Neural Machine Translation (NMT) from the Attention is All You Need paper, with
B: an architecture based on Bi-directional LSTM's in ...
- 11
1
vote
1
answer
50
views
Questions of understanding - Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation
I'm currently analysing the paper Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation (Post, Vilar 2018): https://arxiv.org/abs/1804.06609
I have ...
- 151
0
votes
1
answer
16
views
Training NMT models for noisy social media roman text
I am trying to train an NMT model where the source side is roman text of Asian languages from social media, and target side is English. Note that since roman text is not native to Asia, the ...
- 131
1
vote
2
answers
216
views
Paraphrasing a sentence and changing the tone of it
I am trying to make a model that is capable of translating a sentence into a new and a better form. I would like the model to change the tone and also give it some character. I am using this in my web ...
- 41
3
votes
3
answers
181
views
A good way to organize/store a lot of datasets
In machine translation, we often have bilingual dataset, e.g. for German-English and French-English we will have something that looks like this:
...
- 2,340
1
vote
0
answers
247
views
How to do batch inference on Hugging face pretrained models?
I want to do batch inference on MarianMT model. Here's the code:
...
1
vote
0
answers
24
views
Machine Learning for analyzing and generating sentences from given text inputs
I'm trying to create a program that will translate Sign Language to Text and apply NLP so that the text is understandable to human. I've used CNN for recognizing sign language but I don't know how to ...
0
votes
1
answer
1k
views
Reason for adding 1 to word index for sequence modeling
I notice in many of the tutorials 1 is added to the word_index. For example considering a sample code snippet inspired from <...
- 245
0
votes
1
answer
390
views
Why are mBART50 language codes in an unusual format?
I am trying to use mBART for multilingual translation(about 30 languages) but I am facing an issue with using it as I am currently using langid to identify the languages then load mBART and translate ...
- 213
1
vote
1
answer
162
views
Issue translating large amounts of tweets using Google Translate
I am working on translating large amounts of tweets using this deep-translator which uses the Google Translate API.
Initially everything was fine and tweets were translated with no problems whatsoever ...
- 213
-2
votes
1
answer
33
views
I want to start studying the field of machine translation [closed]
I've studied Japanese language and literature and passed some linguistic courses and now as for my masters, I want to study natural language processing and especially machine translation. so I tried ...
0
votes
1
answer
72
views
How to create a Document Categorization Classifier for different contexts of Documents
I have a doubt solving a test. The idea here is to demonstrate the NLP and Machine Translation abilities.
The Dataset is a multilingual, multi-context set of documents. The dataset is divided on ...
- 3
0
votes
1
answer
25
views
Question of pretraining text-generation task, it seems that pretraining is not work for a small model?
My task is to generate keywords from sentences.
I pretrain a text-generation model. I mask the sentences' tokens and predict the whole sentences' tokens.
Pretraining batch_size = 8 and step = 1000000
...
- 691
0
votes
1
answer
43
views
Passing Dependency/Constituency trees to a Neural Machine Translator
I am working on a project on Neural Machine Translation in the English-Irish domain. I am not an expert and have researched entirely on my own for a technology exhibition so apologies if my question ...
2
votes
0
answers
550
views
What is the difference between register_buffer() and parameter.detach() in PyTorch?
I am writing a PositionalEmbedding() module which is an implementation based on "Attention Is All You Need" using PyTorch. According to the paper, there ...
- 21
1
vote
1
answer
302
views
Unusually High BLEU score on a NMT model
This is the project on Neural Machine Translation on the English/Irish language pair. I have been spending the past month or so trying to train a good baseline to do 'experimentation' on. I have a ...
10
votes
3
answers
6k
views
BPE vs WordPiece Tokenization - when to use / which?
What's the general tradeoff between choosing BPE vs WordPiece Tokenization? When is one preferable to the other? Are there any differences in model performance between the two? I'm looking for a ...
- 238
1
vote
1
answer
58
views
Attention network without hidden state?
I was wondering how useful the encoder's hidden state is for an attention network. When I looked into the structure of an attention model, this is what I found a model generally looks like:
...
- 111
0
votes
1
answer
56
views
Predicting correct match of French to English food descriptions
I have a training and test set of food descriptions pairs (please, see example below)
First name in a pair is a name of food in French
and second word is this food description in English.
Traing set ...
- 295
1
vote
0
answers
34
views
Which is the best way to install Moses for SMT?
I have tried many ways but it always ended up in error. My Configuration: Ubuntu 16.04, Python 3.6.10 The error that I am unable to solve is: undefined reference to
...
- 111
0
votes
1
answer
355
views
Feeding XLM-R embeddings to neural machine translation?
I’m very new to the field of deep learning. My aim is to make a translation between Catalan to Catalan Sign Language. The grammar of the two languages is different
Input: He sells food.
Output (...
- 31
0
votes
1
answer
869
views
Transformer seq2seq model and loading embeddings from XLM-RoBERTa
Is it possible to feed embeddings from XLM- RoBERTa to transformer seq2seq model? I'm working on NMT that translates verbal language sentences to sign language sentences (e.g Input: He sells food. ...
- 31
1
vote
0
answers
41
views
Seq2Seq for sentence correction
I have a task in hand where I get a dirty formed sentence and need to correct it. Examples are, "StackOverflow best question answering platform" to be converted to "StackOverflow is best question ...
- 894
1
vote
2
answers
1k
views
Language translation with convolutional neural network
Many examples of language translation neural networks:
"the cat sat on the mat" -> [model] -> "le chat etait assis sur le tapis"
use RNN, and in particular LSTM. See for example Sentences language ...
- 160
1
vote
0
answers
195
views
How can I finetune XLM-R for neural machine translation between the same language(Catalan to Catalan-with different grammar structure))?
Can I fine-tune XLM-R to do something similar to https://www.guru99.com/seq2seq-model.html
(encoder-decoder architecture or any other way that perform good in small dataset - around 1000)
For example:
...
- 31
2
votes
1
answer
2k
views
Can I fine-tune BERT, ELMO or XLnet for Seq2Seq neural machine translation?
I'm working on neural machine translator that translates English sentences to American sign language sentences(e.g below). I've a quite small dataset - around 1000 sentence pairs. I'm wondering if it ...
- 31
21
votes
3
answers
5k
views
What is the bleu score of professional human translators?
Machine translation models are usually evaluated using bleu score. I want to get some intuition for this score. What is the bleu score of professional human translator?
I know it depends on the ...
- 786
0
votes
1
answer
2k
views
Is it possible feed BERT to seq2seq encoder/decoder NMT (for low resource language)?
I'm working on NMT model which the input and the target sentences are from the same language (but the grammar differs). I'm planning to pre-train and use BERT since I'm working on small dataset and ...
- 31
4
votes
2
answers
244
views
Sentences language translation with neural network, with a simple layer structure (if possible sequential)
Context: Many language sentences translation systems (e.g. French to English) with neural networks use a seq2seq structure:
"the cat sat on the mat" -> [Seq2Seq ...
- 160
24
votes
7
answers
23k
views
Why is the decoder not a part of BERT architecture?
I can't see how BERT makes predictions without using a decoder unit, which was a part of all models before it including transformers and standard RNNs. How are output predictions made in the BERT ...
1
vote
1
answer
103
views
Should the data be shuffled on a translation dataset
As reference to Why should the data be shuffled for machine learning tasks I was curious if this is the case for neural-machine-translation.
I would like to analyze why I think this is the case for ...
- 13
2
votes
1
answer
4k
views
How can I feed BERT to neural machine translation?
I am trying to feed the input and target sentences to an NMT model, I am trying to use BERT here, But I don't have any idea how ...
- 41
2
votes
0
answers
475
views
Weight matrices in transformers
I am trying to understand the transformer architecture.
I am aware that the encoder/decoder contains multiple stacked self attention layers. Further each layer contains multiple heads. For example ...
- 161