Questions tagged [machine-translation]
The machine-translation tag has no usage guidance.
63
questions
-2
votes
1answer
26 views
I want to start studying the field of machine translation [closed]
I've studied Japanese language and literature and passed some linguistic courses and now as for my masters, I want to study natural language processing and especially machine translation. so I tried ...
0
votes
1answer
20 views
How to create a Document Categorization Classifier for different contexts of Documents
I have a doubt solving a test. The idea here is to demonstrate the NLP and Machine Translation abilities.
The Dataset is a multilingual, multi-context set of documents. The dataset is divided on ...
0
votes
1answer
12 views
Question of pretraining text-generation task, it seems that pretraining is not work for a small model?
My task is to generate keywords from sentences.
I pretrain a text-generation model. I mask the sentences' tokens and predict the whole sentences' tokens.
Pretraining batch_size = 8 and step = 1000000
...
0
votes
1answer
27 views
Passing Dependency/Constituency trees to a Neural Machine Translator
I am working on a project on Neural Machine Translation in the English-Irish domain. I am not an expert and have researched entirely on my own for a technology exhibition so apologies if my question ...
0
votes
0answers
92 views
What is the difference between register_buffer() and parameter.detach() in PyTorch?
I am writing a PositionalEmbedding() module which is an implementation based on "Attention Is All You Need" using PyTorch. According to the paper, there ...
1
vote
1answer
57 views
Unusually High BLEU score on a NMT model
This is the project on Neural Machine Translation on the English/Irish language pair. I have been spending the past month or so trying to train a good baseline to do 'experimentation' on. I have a ...
2
votes
0answers
378 views
BPE vs WordPiece Tokenization - when to use / which?
What's the general tradeoff between choosing BPE vs WordPiece Tokenization? When is one preferable to the other? Are there any differences in model performance between the two? I'm looking for a ...
0
votes
1answer
23 views
Attention network without hidden state?
I was wondering how useful the encoder's hidden state is for an attention network. When I looked into the structure of an attention model, this is what I found a model generally looks like:
...
0
votes
1answer
45 views
Predicting correct match of French to English food descriptions
I have a training and test set of food descriptions pairs (please, see example below)
First name in a pair is a name of food in French
and second word is this food description in English.
Traing set ...
0
votes
0answers
10 views
Which is the best way to install Moses for SMT?
I have tried many ways but it always ended up in error. My Configuration: Ubuntu 16.04, Python 3.6.10 The error that I am unable to solve is: undefined reference to
...
0
votes
1answer
187 views
Feeding XLM-R embeddings to neural machine translation?
I’m very new to the field of deep learning. My aim is to make a translation between Catalan to Catalan Sign Language. The grammar of the two languages is different
Input: He sells food.
Output (...
0
votes
1answer
240 views
Transformer seq2seq model and loading embeddings from XLM-RoBERTa
Is it possible to feed embeddings from XLM- RoBERTa to transformer seq2seq model? I'm working on NMT that translates verbal language sentences to sign language sentences (e.g Input: He sells food. ...
1
vote
0answers
22 views
Seq2Seq for sentence correction
I have a task in hand where I get a dirty formed sentence and need to correct it. Examples are, "StackOverflow best question answering platform" to be converted to "StackOverflow is best question ...
0
votes
0answers
51 views
Seeking your advice on XLM-R for NMT
I want to use XLM-R for neural machine translation b/n the same low resource language?
For example: Input-> He sells food(in Catalan) Output-> Food he sells(in Catalan)
Anyone having code example/...
1
vote
2answers
113 views
Language translation with convolutional neural network
Many examples of language translation neural networks:
"the cat sat on the mat" -> [model] -> "le chat etait assis sur le tapis"
use RNN, and in particular LSTM. See for example Sentences language ...
1
vote
0answers
114 views
How can I finetune XLM-R for neural machine translation between the same language(Catalan to Catalan-with different grammar structure))?
Can I fine-tune XLM-R to do something similar to https://www.guru99.com/seq2seq-model.html
(encoder-decoder architecture or any other way that perform good in small dataset - around 1000)
For example:
...
2
votes
1answer
721 views
Can I fine-tune BERT, ELMO or XLnet for Seq2Seq neural machine translation?
I'm working on neural machine translator that translates English sentences to American sign language sentences(e.g below). I've a quite small dataset - around 1000 sentence pairs. I'm wondering if it ...
20
votes
2answers
3k views
What is the bleu score of professional human translators?
Machine translation models are usually evaluated using bleu score. I want to get some intuition for this score. What is the bleu score of professional human translator?
I know it depends on the ...
0
votes
1answer
696 views
Is it possible feed BERT to seq2seq encoder/decoder NMT (for low resource language)?
I'm working on NMT model which the input and the target sentences are from the same language (but the grammar differs). I'm planning to pre-train and use BERT since I'm working on small dataset and ...
3
votes
2answers
98 views
Sentences language translation with neural network, with a simple layer structure (if possible sequential)
Context: Many language sentences translation systems (e.g. French to English) with neural networks use a seq2seq structure:
"the cat sat on the mat" -> [Seq2Seq ...
1
vote
2answers
2k views
Why is the decoder not a part of BERT architecture?
I can't see how BERT makes predictions without using a decoder unit, which was a part of all models before it including transformers and standard RNNs. How are output predictions made in the BERT ...
1
vote
1answer
33 views
Should the data be shuffled on a translation dataset
As reference to Why should the data be shuffled for machine learning tasks I was curious if this is the case for neural-machine-translation.
I would like to analyze why I think this is the case for ...
1
vote
1answer
2k views
How can I feed BERT to neural machine translation?
I am trying to feed the input and target sentences to an NMT model, I am trying to use BERT here, But I don't have any idea how ...
1
vote
0answers
112 views
Weight matrices in transformers
I am trying to understand the transformer architecture.
I am aware that the encoder/decoder contains multiple stacked self attention layers. Further each layer contains multiple heads. For example ...
1
vote
2answers
61 views
where to start in natural language processing for a language
My native language is a regional language and few people speak it. I have some assignements in a machine learning course and i was thinking about doing some natural languge processing on my native ...
0
votes
1answer
283 views
Help making a custom categorical loss function in Keras
I am a bit new to machine learning, and I'm trying to get the basics working towards a bigger project using a very simple encoder-decoder model. It looks like this:
...
1
vote
0answers
28 views
Can BERT embeddings be used to reproduce the original content of the text?
From what I understand, BERT provides contextualized embeddings that are not deterministic the way Word2Vec embeddings (i.e. the word "Queen" doesn't always produce the same vector, it'll be different ...
0
votes
0answers
16 views
searching for vocabulary for machine translation
Searching for DE-EN vocabulary, that I can use in my machine translation.
It should consist of DE-EN phrases, words, sentences etc.
Got one from here http://www.manythings.org/anki/ , but searching ...
1
vote
3answers
795 views
Using Google Translate API to create a Translation Dataset
Is it a good idea? ;-)
Is it legal to do so? Is it legal to release such a dataset to public?
Say I have a language X for which I want to create a dataset for translation to/from English, for which I ...
0
votes
1answer
156 views
NMT, What if we do not pass input for decoder?
For transformer-based neural machine translation (NMT), take English-Chinese for example, we pass English for encoder and use decoder input(Chinese) attend to encoder output, then final output.
What ...
3
votes
0answers
120 views
What is Bit Per Character?
What is Bits per Character (bpc) metric which has been used to measure the model accuracy with reference to text8 and ...
1
vote
1answer
86 views
Why is hard for neural machine translation model to learn rare words?
i'm kinda new with neural machine translation. I've read some papers, authors usually limit the size of vocabulary by replace rare words by unk token. In this paper, they said that "...NMT model ...
1
vote
1answer
31 views
Is it normal when BLEU score on filtered data by length is greater than BLEU score on whole data
I am creating 2 neural machine translation model (model A and B with different improvements each model) with fairseq-py. When I evaluate model with bleu score, model A BLEU score is 25.9 and model C ...
6
votes
1answer
798 views
Multi-Head attention mechanism in transformer and need of feed forward neural network
After reading the paper, Attention is all you need, I have two questions:
1. What is the need of a multi-head attention mechanism?
The paper says that:
"Multi-head attention allows the model to ...
3
votes
0answers
25 views
Back-Translation model for German and English
Do you know of any pre-trained models for back translation between German and English? I am aware that there are ways to include a monolingual corpus into the training of a machine translation model (...
1
vote
2answers
158 views
Embedding Values in word2vec
Are the embedding values for a particular word using word2vec Skipgram model the weights of the first layer or the softmax output of the function?
Does the embedding value change according to the ...
0
votes
1answer
46 views
What is used for Machine Translation besides RNN
I am doing a university report and it seems that encode-decode RNN are optimal for Machine Translation. I need something else to compare it to but I can't seem to make a proper google search for it.
...
1
vote
1answer
520 views
How to build a machine translation system for a new language
I am trying to figure out what are the options for building a natural language translation model for a language that is not yet supported by existing machine translations.
The project is to build a ...
2
votes
1answer
304 views
Implementing back translation as a data augmentation for text classification
Since back translation English->other language -> English seems like quite a useful data augmentation technique , I wanted to experiment with it. E.g. it occurred to me that languages from very ...
2
votes
1answer
518 views
How to create a language translator from scratch?
I want to create a translator which can translate English, Korean and Tamil sentences into English sentence, I tried googletrans but is there any way to create something better than that using DL and ...
3
votes
0answers
693 views
How to train tensorflow's transformer model on my own data?
https://github.com/tensorflow/models/blob/master/official/transformer has an implementation of transformer model.
I want to train the model on my own data(consisting of two files, src.txt, and tgt.txt)...
2
votes
1answer
34 views
Statistical machine translation word alignment for FR-ENG and ENG-FR: what is p(e) and p(f)?
I'm currently trying to implement this paper, but am struggling to understand some of the math here. I'm pretty sure I understand how to implement the E-step, but for the M-step, I'm confused on how ...
2
votes
1answer
243 views
word/sentence alignment for English document
I have a English document, which is preprocessed into two versions. I want to align words or sentences from these two versions of document. A simple example is as below:
...
2
votes
1answer
1k views
Is there a dataset with news articles and their headlines? [closed]
I need a set of news headlines and articles to help me in a project on automatic summarization. Is there such a dataset or something similar?
3
votes
0answers
109 views
Encoder-Decoder Sequence-to-Sequence Model for Translations in Both Directions
Is it possible to use a pre-trained sequence to sequence encoder-decoder model which translates an input text in source language to an output in target language to do an inverse translation? That is, ...
1
vote
0answers
18 views
How can we predict Geo Tagged Pollution Parameters using a Machine Learning Model with an Android Device?
Newly installed Pollution Control Sensory Platforms are dumping Continous/Discrete Environment Pollution Parametric Data from various locations.
Whats the best format structured or Unstructured of ...
0
votes
1answer
959 views
Do we really need `<unk>` tokens?
I am wondering, do we really need <unk> tokens?
Why do we limit our vocabulary?
Is it for speed? Accuracy?
If we disable all limitations, what do you ...
5
votes
2answers
7k views
Is there “Attention Is All You Need” implementation in Keras?
Has anyone seen this model's implementation using Keras?
inb4: tensorflow, pytorch
3
votes
1answer
5k views
Problem with keras model loading
I want to use this keras seq2seq example to train my model. But i dont undestand role of encoder and decoder model and why we dont use model which we trained here.
model.fit([encoder_input_data, ...
1
vote
0answers
110 views
Algorithm to parse PSD into html/XML?
I have been working on a project and we were trying to convert a PSD file to a HTML for web applications as well as a Layout XML for android. We worked our way to generate basic skeletal html/xml but ...
