Questions tagged [sequence-to-sequence]
The sequence-to-sequence tag has no usage guidance.
9
questions
16
votes
3
answers
34k
views
How to determine feature importance in a neural network?
I have a neural network to solve a time series forecasting problem. It is a sequence-to-sequence neural network and currently it is trained on samples each with ten features. The performance of the ...
- 418
7
votes
1
answer
5k
views
Minimal working example or tutorial showing how to use Pytorch's nn.TransformerDecoder for batch text generation in training and inference modes?
I want to solve a sequence-to-sequence text generation task (e.g. question answering, language translation, etc.).
For the purposes of this question, you may assume that I already have the input part ...
- 377
15
votes
1
answer
10k
views
Why do we need to add START <s> + END </s> symbols when using Recurrent Neural Nets for Sequence-to-Sequence Models?
In the Sequence-to-Sequence models, we often see that the START (e.g. <s>) and END (e.g. </s>) symbols are added to ...
- 2,330
4
votes
2
answers
221
views
Sentences language translation with neural network, with a simple layer structure (if possible sequential)
Context: Many language sentences translation systems (e.g. French to English) with neural networks use a seq2seq structure:
"the cat sat on the mat" -> [Seq2Seq ...
- 150
3
votes
1
answer
1k
views
How does the Transformer predict n steps into the future?
I have barely been able to find an implementation of the Transformer (that is not bloated nor confusing), and the one that I've used as reference was the PyTorch implementation. However, the Pytorch ...
- 33
3
votes
1
answer
55
views
String together a set of tokens into a sequence
I have this problem scenario - Given a set of tokens, string them or a subset of the tokens together using stop words into a sequence. I am clear that I can have potentially infinite pre-training data ...
- 141
2
votes
0
answers
1k
views
Recommended model for univariate or multivariate multistep ahead time series forecasting
I have a dataset consisting of recurring and non-recurring expense transactions from bank accounts, as well as other features describing the bank account and each transation. I aggregate these ...
- 189
1
vote
1
answer
155
views
Why does a decoder generate all hidden states during inference?
Seems that in Vanilla transformers at least (a la AIAYN), during inference time, the hidden states are generated for all tokens in the input sequence, but only the last one is used to predict the next ...
- 163
0
votes
1
answer
57
views
How to make a pipeline for Videos Dataset for TensorFlow [Sequence Matters] & train Model Effectively with Low Memory System
I am working on a Deep Learning project and I am facing an issue with the size of the dataset. I want to make a pipeline for video dataset [Sequence Matters]. Because if I try the load the whole ...
- 21