Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [machine-translation]

Machine translation in a data science context refers to the process of using machine learning techniques to translate input provided in one language into output in another. It includes topics referring to using text/corpus data, neural machine translation (NMT), deep learning, and speech recognition.

Filter by
Sorted by
Tagged with
2 votes
0 answers
54 views

Why does cross-attention in an NMT decoder use the encoder embeddings as values?

In the Vaswani 2017 paper introducing encoder-decoder transformers, the cross-attention step in the decoder is visualised as follows: Because keys and values are always taken to be equal, this figure ...
Mew's user avatar
  • 233
1 vote
1 answer
75 views

Unsupervised Machine Translation System Using Variational Autoencoder Models

I want to work on an unsupervised machine translation system using a variational autoencoder. I did a literature review but didn't find any related work, and most of the work is based on denoising ...
kartikeya saraswat's user avatar
0 votes
2 answers
53 views

Why are there no automated translated subtitles?

This might be a rather naive question. There are programs that can automate subtitles, and also programs that can automate translations. Why, then, is there apparently no program that can automate ...
Allure's user avatar
  • 275
0 votes
0 answers
8 views

Systematic way of selecting internet texts for a machine translation corpus / dataset?

I am currently working on a neural machine translation project and want to gather a corpus (or dataset) of internet texts that are written in standard and plain language. In theory, it certainly makes ...
tschomacker's user avatar
0 votes
1 answer
25 views

How to evaluate machine translations of long documents?

I'm using Python and I want to compare the output of two machine-translation (ish) systems. Most of the tools seem to be focused on sentence-by-sentence evaluation. Either I get memory blow-ups with ...
Paul Prescod's user avatar
0 votes
0 answers
25 views

Extremely silly transformers fine-tuning doubt

Suppose say I have a TFT5ConditionalGenration model instantiated with tensor flow. When you do model.fit, the model will pass the inputs in a feed-forward way similar to a call method. The model will ...
NeverGiveUp's user avatar
0 votes
0 answers
631 views

How to Fine-Tune Multilingual Translational Transformer Models

I'm trying to fine-tune facebook/nllb-200-distteled-600M model on my english to arabic dataset, but the results are so bad I'm expecting there should be something wrong with defining the model and ...
Mohamed Abduljawad's user avatar
2 votes
0 answers
55 views

How much data and computation power do I need to train a machine translation model using Transformer architecture?

I am working right now on creating a dataset to use in creating a machine translation model to translate between two dialects. I have two questions that I am trying to find an answer for: How much ...
user13288833's user avatar
0 votes
1 answer
673 views

How to translate text automatically using Google Translate API (or any other approach) in python

I have a dataset of reviews from TripAdvisor and I would like to translate non-English reviews into English. The reviews are in many different languages: my dataset contains reviews in 44 different ...
Alberto De Benedittis's user avatar
0 votes
0 answers
36 views

Is it possible for computers to tell how many people are speaking in a audio recording

Suppose I had a audio recording of 15 students saying "Here" all at the same. Can I tell how many students were speaking and who they were using machine learning? I want to create a school ...
Mustafa's user avatar
  • 101
0 votes
1 answer
202 views

Extract the embedding from a specific layer of MarianModel

I am using using MarianModel from the hub of HuggingFace for a translation task. Now I want to extract the embedding from the output of the last MarianEncoderLayer ...
lenhhoxung's user avatar
1 vote
1 answer
77 views

What could cause pre-trained Opus-MT models have wildly varying inference time when being used with transformers library?

I have been testing pre-trained Opus-MT models ported to transformers library for python implementation. Specifically, I am using opus-mt-en-fr for English to French translation. And the tokenizer and ...
Shan Dou's user avatar
  • 131
0 votes
2 answers
146 views

What is meant by averaging inhibits it in the paper 'Attention is All You Need'?

Could anyone explain to me about the sentence below? What is meant by averaging inhibits it? Multi-head attention allows the model to jointly attend to information from different representation ...
Jayden's user avatar
  • 1
0 votes
1 answer
361 views

WMT: What are the differences of WMT14, WMT15 and WMT16 datasets?

Each year, the Workshop on Statistical Machine Translation (WMT) holds a conference that focuses on new tasks, papers, and findings in the field of machine translation. Let's say we are talking about ...
Ramón Wilhelm's user avatar
0 votes
1 answer
199 views

For an LSTM-based seq2seq model, is reversing the input still necessary or advised when using attention?

The original seq2seq paper reversed the input sequence and cited multiple reasons for doing so. See: Why does LSTM performs better when the source target is reversed? (Seq2seq) But when using ...
Hank's user avatar
  • 3
1 vote
0 answers
2k views

Pytorch build_vocab_from_iterator giving vocabulary with very few words

I am trying to build a translation model in pytorch. Following this post on pytorch I downloaded the multi30k dataset and spacy models for English and German. ...
k-c's user avatar
  • 13
1 vote
0 answers
83 views

Why does Bahdanau Attention Have to be Causal?

Using the Bahdanau attention layer on Tensorflow for time series prediction, although conceptually it is similar to NLP applications. This is how the minimal example code for a single layer looks like....
Della's user avatar
  • 335
1 vote
2 answers
2k views

Self Attention vs LSTM with Attention for NMT

I am trying to compare the A: Transformer-based architecture for Neural Machine Translation (NMT) from the Attention is All You Need paper, with B: an architecture based on Bi-directional LSTM's in ...
mashrivas's user avatar
1 vote
1 answer
54 views

Questions of understanding - Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation

I'm currently analysing the paper Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation (Post, Vilar 2018): https://arxiv.org/abs/1804.06609 I have ...
Ramón Wilhelm's user avatar
0 votes
1 answer
16 views

Training NMT models for noisy social media roman text

I am trying to train an NMT model where the source side is roman text of Asian languages from social media, and target side is English. Note that since roman text is not native to Asia, the ...
Gokul NC's user avatar
  • 131
1 vote
2 answers
269 views

Paraphrasing a sentence and changing the tone of it

I am trying to make a model that is capable of translating a sentence into a new and a better form. I would like the model to change the tone and also give it some character. I am using this in my web ...
Loukik's user avatar
  • 41
3 votes
3 answers
187 views

A good way to organize/store a lot of datasets

In machine translation, we often have bilingual dataset, e.g. for German-English and French-English we will have something that looks like this: ...
alvas's user avatar
  • 2,410
1 vote
0 answers
266 views

How to do batch inference on Hugging face pretrained models?

I want to do batch inference on MarianMT model. Here's the code: ...
sai_varshittha's user avatar
1 vote
0 answers
24 views

Machine Learning for analyzing and generating sentences from given text inputs

I'm trying to create a program that will translate Sign Language to Text and apply NLP so that the text is understandable to human. I've used CNN for recognizing sign language but I don't know how to ...
Lord Dickenstein's user avatar
0 votes
1 answer
2k views

Reason for adding 1 to word index for sequence modeling

I notice in many of the tutorials 1 is added to the word_index. For example considering a sample code snippet inspired from <...
data_person's user avatar
0 votes
1 answer
425 views

Why are mBART50 language codes in an unusual format?

I am trying to use mBART for multilingual translation(about 30 languages) but I am facing an issue with using it as I am currently using langid to identify the languages then load mBART and translate ...
yudhiesh's user avatar
  • 213
1 vote
1 answer
173 views

Issue translating large amounts of tweets using Google Translate

I am working on translating large amounts of tweets using this deep-translator which uses the Google Translate API. Initially everything was fine and tweets were translated with no problems whatsoever ...
yudhiesh's user avatar
  • 213
-2 votes
1 answer
33 views

I want to start studying the field of machine translation [closed]

I've studied Japanese language and literature and passed some linguistic courses and now as for my masters, I want to study natural language processing and especially machine translation. so I tried ...
takeshi hatake's user avatar
0 votes
1 answer
75 views

How to create a Document Categorization Classifier for different contexts of Documents

I have a doubt solving a test. The idea here is to demonstrate the NLP and Machine Translation abilities. The Dataset is a multilingual, multi-context set of documents. The dataset is divided on ...
Marc's user avatar
  • 3
0 votes
1 answer
25 views

Question of pretraining text-generation task, it seems that pretraining is not work for a small model?

My task is to generate keywords from sentences. I pretrain a text-generation model. I mask the sentences' tokens and predict the whole sentences' tokens. Pretraining batch_size = 8 and step = 1000000 ...
CoderOnly's user avatar
  • 711
0 votes
1 answer
44 views

Passing Dependency/Constituency trees to a Neural Machine Translator

I am working on a project on Neural Machine Translation in the English-Irish domain. I am not an expert and have researched entirely on my own for a technology exhibition so apologies if my question ...
Justin Cunningham's user avatar
2 votes
0 answers
586 views

What is the difference between register_buffer() and parameter.detach() in PyTorch?

I am writing a PositionalEmbedding() module which is an implementation based on "Attention Is All You Need" using PyTorch. According to the paper, there ...
Jason Young's user avatar
1 vote
1 answer
318 views

Unusually High BLEU score on a NMT model

This is the project on Neural Machine Translation on the English/Irish language pair. I have been spending the past month or so trying to train a good baseline to do 'experimentation' on. I have a ...
Justin Cunningham's user avatar
10 votes
3 answers
7k views

BPE vs WordPiece Tokenization - when to use / which?

What's the general tradeoff between choosing BPE vs WordPiece Tokenization? When is one preferable to the other? Are there any differences in model performance between the two? I'm looking for a ...
vgoklani's user avatar
  • 238
1 vote
1 answer
61 views

Attention network without hidden state?

I was wondering how useful the encoder's hidden state is for an attention network. When I looked into the structure of an attention model, this is what I found a model generally looks like: ...
JMRC's user avatar
  • 111
0 votes
1 answer
56 views

Predicting correct match of French to English food descriptions

I have a training and test set of food descriptions pairs (please, see example below) First name in a pair is a name of food in French and second word is this food description in English. Traing set ...
dokondr's user avatar
  • 295
1 vote
0 answers
36 views

Which is the best way to install Moses for SMT?

I have tried many ways but it always ended up in error. My Configuration: Ubuntu 16.04, Python 3.6.10 The error that I am unable to solve is: undefined reference to ...
Abdullah Khilji's user avatar
0 votes
1 answer
364 views

Feeding XLM-R embeddings to neural machine translation?

I’m very new to the field of deep learning. My aim is to make a translation between Catalan to Catalan Sign Language. The grammar of the two languages is different Input: He sells food. Output (...
NLP Dude's user avatar
0 votes
1 answer
916 views

Transformer seq2seq model and loading embeddings from XLM-RoBERTa

Is it possible to feed embeddings from XLM- RoBERTa to transformer seq2seq model? I'm working on NMT that translates verbal language sentences to sign language sentences (e.g Input: He sells food. ...
NLP Dude's user avatar
1 vote
0 answers
41 views

Seq2Seq for sentence correction

I have a task in hand where I get a dirty formed sentence and need to correct it. Examples are, "StackOverflow best question answering platform" to be converted to "StackOverflow is best question ...
Sandeep Bhutani's user avatar
1 vote
3 answers
1k views

Language translation with convolutional neural network

Many examples of language translation neural networks: "the cat sat on the mat" -> [model] -> "le chat etait assis sur le tapis" use RNN, and in particular LSTM. See for example Sentences language ...
Basj's user avatar
  • 160
1 vote
0 answers
196 views

How can I finetune XLM-R for neural machine translation between the same language(Catalan to Catalan-with different grammar structure))?

Can I fine-tune XLM-R to do something similar to https://www.guru99.com/seq2seq-model.html (encoder-decoder architecture or any other way that perform good in small dataset - around 1000) For example: ...
NLP Dude's user avatar
2 votes
1 answer
2k views

Can I fine-tune BERT, ELMO or XLnet for Seq2Seq neural machine translation?

I'm working on neural machine translator that translates English sentences to American sign language sentences(e.g below). I've a quite small dataset - around 1000 sentence pairs. I'm wondering if it ...
NLP Dude's user avatar
21 votes
3 answers
5k views

What is the bleu score of professional human translators?

Machine translation models are usually evaluated using bleu score. I want to get some intuition for this score. What is the bleu score of professional human translator? I know it depends on the ...
Amit Keinan's user avatar
0 votes
1 answer
2k views

Is it possible feed BERT to seq2seq encoder/decoder NMT (for low resource language)?

I'm working on NMT model which the input and the target sentences are from the same language (but the grammar differs). I'm planning to pre-train and use BERT since I'm working on small dataset and ...
NLP Dude's user avatar
4 votes
2 answers
257 views

Sentences language translation with neural network, with a simple layer structure (if possible sequential)

Context: Many language sentences translation systems (e.g. French to English) with neural networks use a seq2seq structure: "the cat sat on the mat" -> [Seq2Seq ...
Basj's user avatar
  • 160
29 votes
7 answers
25k views

Why is the decoder not a part of BERT architecture?

I can't see how BERT makes predictions without using a decoder unit, which was a part of all models before it including transformers and standard RNNs. How are output predictions made in the BERT ...
Hrishikesh Athalye's user avatar
1 vote
1 answer
109 views

Should the data be shuffled on a translation dataset

As reference to Why should the data be shuffled for machine learning tasks I was curious if this is the case for neural-machine-translation. I would like to analyze why I think this is the case for ...
user86834's user avatar
2 votes
1 answer
4k views

How can I feed BERT to neural machine translation?

I am trying to feed the input and target sentences to an NMT model, I am trying to use BERT here, But I don't have any idea how ...
Hamed's user avatar
  • 39
2 votes
0 answers
525 views

Weight matrices in transformers

I am trying to understand the transformer architecture. I am aware that the encoder/decoder contains multiple stacked self attention layers. Further each layer contains multiple heads. For example ...
boredaf's user avatar
  • 161