All Questions
7 questions
1
vote
1
answer
154
views
Inference order in BERT masking task
In BERT, multiple words in a single sentence can be masked at once. Does the model infer all of those words at once or iterate over them in either left to right or some other order?
For example:
The ...
0
votes
1
answer
125
views
Optimal input setup for character-level text classification RNN
I want to classify 500-character long text samples as to whether they look like natural language using a character-level RNN. I'm unsure as to the best way to feed the input to the RNN. Here are two ...
- 101
0
votes
1
answer
379
views
Feeding XLM-R embeddings to neural machine translation?
I’m very new to the field of deep learning. My aim is to make a translation between Catalan to Catalan Sign Language. The grammar of the two languages is different
Input: He sells food.
Output (...
- 31
1
vote
0
answers
28
views
Skip-gram trained on The Hobbit: no improvement in the similarity of the word representation
I've trained a simple skipgram NNLM (window size = 5) on The Hobbit. This is the rough pseudocode:
...
- 777
1
vote
0
answers
100
views
Why is MLP working similar to RNN for text generation
I was trying to perform text generation using only a character level feed-forward neural network after having followed this tutorial which uses LSTM. I one-hot encoded the characters of my corpus ...
- 183
1
vote
1
answer
67
views
The principle of LM deep model
Language model(LM) is the task of predicting the next word.
Does the deep model need the encoder? From the ptb code of tensor2tensor, I find the deep model do not contains the encoder.
Or both with-...
- 711
3
votes
1
answer
802
views
Neural Networks for Predictive typing
I don't have a background in neural networks. But, various studies has been proved that neural networks (feed forward / Recurrent) outperformed n-gram language modeling for predicting words in a ...
- 33