Questions tagged [sequence-to-sequence]
The sequence-to-sequence tag has no usage guidance.
44
questions
0
votes
1answer
5 views
Confusion about Decoder labels for training seq-to-seq models
So in seq-to-seq models for say NMT, the decoder is a sequence model for the right-shifted intended output. My question is, during training, are the inputs and outputs of the decoder supposed to be ...
1
vote
0answers
38 views
Why isnt my seq2seq model reconising what the <END> tag is?
I am making a seq2seq chatbot with the help of this guide: https://medium.com/predict/creating-a-chatbot-from-scratch-using-keras-and-tensorflow-59e8fc76be79 . I set up the model and data processing ...
0
votes
0answers
8 views
Encoding numbers and words
I am fairly new to seq2seq models in nlp and just really learned about them. Anyway, in many of the examples, I have seen there has been to approaches to providing a model with data. One in which is ...
0
votes
0answers
13 views
Test case generation from user stories using NLP and NLG
I have an existing database with User stories (with multiple text fields like Description with Use cases, acceptance criteria etc) and corresponding test cases with steps for each test case.
I am ...
0
votes
0answers
22 views
Which algorithm to use for path to prescription?
I am working on a business problem in the commercial pharma industry. In the pharma industry, we have medical representatives (Reps) selling drugs to health care providers (HCPs). They frequently ...
0
votes
0answers
31 views
Error propagation in Time series forecast with many-to-many multi-steps RNN/LSTM
I am trying to do a many-to-many time series forecast, which features an encoder-decoder model to predict with variable input and fixed prediction period. In my case, I want to predict for the future ...
0
votes
0answers
13 views
Deep Learning - Predict the relative order of data
I am facing a problem in which i want to predict the order of data. I was searching for research papers, however i do not know how this problem is named in academia.
I encountered the following well ...
0
votes
1answer
20 views
Seq2Seq Model training: Encoder vs. Decoder
Can someone point me to an article which explains how the model training is done in Seq2Seq? I know "Teacher Forcing" is used but what I found so far hasn't been detailed enough. What I am most ...
1
vote
1answer
325 views
IndexError: list index out of range
I'm implementing a sequence-2-sequence model with RNN-VAE architecture, and I use an attention mechanism. I have problem in the decoder part.
I'm struggling with this error: IndexError: list index ...
0
votes
0answers
38 views
ValueError: Dimensions must be equal, but are 256 and 12 for 'attention_layer/MatMul_1' (op: 'MatMul') with input shapes: [?,256], [12,256]
I'm working on a sequence-to-sequence approach using LSTM and a VAE with an attention mechanism.
...
0
votes
0answers
186 views
ValueError: Cannot convert a partially known TensorShape to a Tensor: (?, 256)
I'm working on a sequence to sequence approach using LSTM and a VAE with an attention mechanism.
...
0
votes
0answers
18 views
Strategy for “forcing” number of labels in seq2seq predictions with Keras?
I'm trying to train a seq2seq model that for every timestep in a given timeseries sample will output 1 of 6 possible labels.
Furthermore, the training data is constructed in such a way that
Each ...
0
votes
0answers
26 views
What encoding to use for my musical vectors?
I'm trying to build a music recommendations system using an encoder-decoder sequence-to-sequence architecture using keras. My dataset comprises of playlists containing songs represented as a 13-...
1
vote
1answer
34 views
Working ofLSTM with multiple Units - NER
I am trying to understand working of LSTM networks and kind of not clear about how different neurons in a cell interact each other. I had a look at a similar question, but still not clear about few ...
0
votes
0answers
91 views
Seq2seq model that gets as input a sentence and outputs the same sentence
I tried to implement a model that takes as input sentences, which are hate_tweets and outputs exactly the same sentences. For this reason, I gave Input to the encoder and decoder exactly the same ...
0
votes
0answers
49 views
Using VAE with Sequence to Sequence Approach
In the code below, I'm using a VAE with a seq-to-seq approach for translation. At the beginning I sarted only by using a simple seq-to-seq approach which implements a RNN-AE, until this step I had not ...
1
vote
0answers
20 views
Adding context in a sequence to sequence problem
The encoder of a seq2seq model is meant to generate a conditioning context for the decoder, as mentioned here
A RNN layer (or stack thereof) acts as "encoder": it processes the input sequence and ...
3
votes
1answer
737 views
One-hot encode multi-class multi-label sequences
Suppose I want to build a timeseries where each timestep is represented by a categorical array: the encoded sequences look like [[2, 0, 5],[3, 1, 4],..] and each ...
1
vote
0answers
22 views
Initialising states in a multilayer sequence to sequence model
With a sequence to sequence model where the enocoder and decoder are both comprised of one layer each, the initial state of the decoder is initialised to use the final states of the encoder layer.
In ...
1
vote
0answers
49 views
What should the size of the decoder output be in a sequence to sequence model
In a sequence to sequence model, a lot of the tutorials I have read state that the decoder target length should be the same as the encoder input length (https://blog.keras.io/building-autoencoders-in-...
2
votes
1answer
3k views
How to determine feature importance in a neural network?
I have a neural network to solve a time series forecasting problem. It is a sequence-to-sequence neural network and currently it is trained on samples each with ten features. The performance of the ...
2
votes
0answers
368 views
Recommended model for univariate or multivariate multistep ahead time series forecasting
I have a dataset consisting of recurring and non-recurring expense transactions from bank accounts, as well as other features describing the bank account and each transation. I aggregate these ...
1
vote
1answer
727 views
Predicting next number in a sequence - data analysis
I am a machine learning newbie and I am working on a project where I'm given a sequence of integers all of which are in the range 0 to 70. My goal is to predict the next integer in the sequence given ...
0
votes
1answer
2k views
Does this encoder-decoder LSTM make sense for time series sequence to sequence?
TASK
given $\vec x = [x_{t=-3}, x_{t=-2}, x_{t=-1}, x_{t=0}]$
predict $\vec y = [x_{t=1}, x_{t=2}]$
Whith an LSTM encoder-decoder (seq2seq)
MODEL
NOTE: the ? symbol in the shape of the tensors ...
1
vote
0answers
838 views
Why does Position Embeddings work?
In the papers "Convolutional Sequence to Sequence Learning" and
"Attention Is All You Need", positions embeddings are simply added to the input words embeddings to give the model a sense of the order ...
1
vote
1answer
311 views
Give Variable Length input to LSTM
My input data consist of list of list. Both list have dynamic length for every example like below.
...
0
votes
1answer
34 views
What type of neural network could emulate a binary to HTML conversion tool?
I've got a problem, which I thought could be solved by using a neural network:
I've got a binary file and a tool that converts that file into a readable html file (probably a text file as well). How ...
1
vote
1answer
34 views
Can Sequence to sequence models be used to convert code from one programming language to another?
Is it possible to convert a code from one language to another using sequence to sequence programming. if not sequence to sequence, which algorithm can i use to do this?
3
votes
3answers
114 views
Predict output sequence one at a time with feedback
I would like to solve the following classification problem: Given an input sequence and zero or more initial terms of the true output, predict the next term in the output sequence.
For example, my ...
3
votes
2answers
233 views
Is this a problem for a Seq2Seq model?
I'm struggling to find a tutorial/example which covers using an seq2seq model for sequential inputs other then text/translation.
I have a multivariate dataset with n number of input variables each ...
2
votes
0answers
97 views
Encoder-Decoder Sequence-to-Sequence Model for Translations in Both Directions
Is it possible to use a pre-trained sequence to sequence encoder-decoder model which translates an input text in source language to an output in target language to do an inverse translation? That is, ...
1
vote
0answers
48 views
On-the-fly seq2seq: starting translation before the input sequence ends
By now anybody even remotely familiar with RNNs has been exposed to the famous figure representing variants of such networks:
However one type that is missing is one where there would be a partial ...
1
vote
0answers
202 views
Neural network outputting same result for all inputs
I'm building an encoder-decoder neural network in Keras for sequence generation. The specific task is to try and change the styles of the text.
Both my encoder and decoders are LSTMs with latent ...
2
votes
1answer
161 views
Keras: Softmax output into embedding layer
I'm trying to build an encoder-decoder network in Keras to generate a sentence of a particular style. As my problem is unsupervised i.e. I don't have the ground truths for the generated sentences, I ...
1
vote
2answers
135 views
What are the benefits and tradeoffs of a 1D conv vs a multi-input seq2seq LSTM model?
I have 6 sequences, s1,..,s6. Using all sequences I want to predict a binary vector q = [0,0,0,1,1,1,0,0,0,1,1,1,...], which is a mask of the activity of the 6 sequences.
I have looked at seq2seq ...
1
vote
1answer
120 views
Training Encoder-Decoder using Decoder Outputs
I am trying to build an encoder-decoder model for a text style transfer problem. The problem is I don't have parallel data between the two styles so I need to train the model in an unsupervised ...
0
votes
1answer
366 views
Do we really need `<unk>` tokens?
I am wondering, do we really need <unk> tokens?
Why do we limit our vocabulary?
Is it for speed? Accuracy?
If we disable all limitations, what do you ...
1
vote
1answer
1k views
Very long sequence in neural networks
Beginner's question regarding sequences in neural networks: suppose I have classification problem that looks like:
X = very long sequence of varying length.
Y = class (assume for simplicity y=0/1).
...
1
vote
2answers
38 views
Group prediction
I have following sort of data coming every day:
(0)(3,4,5)(6,9,1)(5,35,12,232)
(1)(5,1,4)(6,2)(12,54,12,43)(8,23,65)
(2)(6,7,2)(34,3)
(3)(4323,23,12,4543)
(4)(987,32,324,23,224,12,213,21)(1,2)
(...
1
vote
1answer
117 views
How to pad real-valued sequences
I have several sequences of univariate real-valued time-series data. The sequences are of different lengths and right now I cannot batch them and feed them to a network.
What is the correct procedure ...
3
votes
1answer
2k views
How/What to initialize the hidden states in RNN sequence-to-sequence models?
In an RNN sequence-to-sequence model, the encode input hidden states and the output's hidden states needs to be initialized before training.
What values should we initialize them with? How should we ...
6
votes
1answer
539 views
How do attention mechanisms in RNNs learn weights for a variable length input
Attention mechanisms in RNNs are reasonably common to sequence to sequence models.
I understand that the decoder learns a weight vector $\alpha$ which is applied as a weighted sum of the output ...
5
votes
1answer
599 views
Input for LSTM for financial time series directional prediction
I'm working on using an LSTM to predict the direction of the market for the next day.
My question concerns the input for the LSTM. My data is a financial time series $x_1 \ldots x_t$ where each $x_i$...
5
votes
1answer
3k views
Why do we need to add START <s> + END </s> symbols when using Recurrent Neural Nets for Sequence-to-Sequence Models?
In the Sequence-to-Sequence models, we often see that the START (e.g. <s>) and END (e.g. </s>) symbols are added to ...
