Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [sequence-to-sequence]

The tag has no usage guidance.

2
votes
0answers
11 views

Sequence to sequence LSTM is “slow” to observe changes. Can this be improved?

I have some multivariate time series, and I'm trying to teach an LSTM with simulated data what is good and what is bad, so the input is a timeseries with a label for every timestep, i.e. [1,1,1,1,1,1,...
1
vote
0answers
8 views

Initialising states in a multilayer sequence to sequence model

With a sequence to sequence model where the enocoder and decoder are both comprised of one layer each, the initial state of the decoder is initialised to use the final states of the encoder layer. In ...
1
vote
0answers
14 views

What should the size of the decoder output be in a sequence to sequence model

In a sequence to sequence model, a lot of the tutorials I have read state that the decoder target length should be the same as the encoder input length (https://blog.keras.io/building-autoencoders-in-...
1
vote
0answers
48 views

How to determine feature importance in a neural network?

I have a neural network to solve a time series forecasting problem. It is a sequence-to-sequence neural network and currently it is trained on samples each with ten features. The performance of the ...
0
votes
0answers
32 views

Best Architecture for LSTM Network for Stock Prediction

I am building an LSTM model to predict stock prices using TensorFlow. Is it best to structure the model so that it accepts $X=[x_0, x_1, ... x_{n-1}]$ and predicts $y=x_n$, or accepts $X=[x_0, x_1, ......
2
votes
0answers
80 views

Recommended model for univariate or multivariate multistep ahead time series forecasting

I have a dataset consisting of recurring and non-recurring expense transactions from bank accounts, as well as other features describing the bank account and each transation. I aggregate these ...
0
votes
1answer
107 views

Predicting next number in a sequence - data analysis

I am a machine learning newbie and I am working on a project where I'm given a sequence of integers all of which are in the range 0 to 70. My goal is to predict the next integer in the sequence given ...
0
votes
1answer
143 views

Does this encoder-decoder LSTM make sense for time series sequence to sequence?

TASK given $\vec x = [x_{t=-3}, x_{t=-2}, x_{t=-1}, x_{t=0}]$ predict $\vec y = [x_{t=1}, x_{t=2}]$ Whith an LSTM encoder-decoder (seq2seq) MODEL NOTE: the ? symbol in the shape of the tensors ...
1
vote
0answers
128 views

Why does Position Embeddings work?

In the papers "Convolutional Sequence to Sequence Learning" and "Attention Is All You Need", positions embeddings are simply added to the input words embeddings to give the model a sense of the order ...
1
vote
1answer
160 views

Give Variable Length input to LSTM

My input data consist of list of list. Both list have dynamic length for every example like below. ...
0
votes
1answer
20 views

What type of neural network could emulate a binary to HTML conversion tool?

I've got a problem, which I thought could be solved by using a neural network: I've got a binary file and a tool that converts that file into a readable html file (probably a text file as well). How ...
1
vote
1answer
24 views

Can Sequence to sequence models be used to convert code from one programming language to another?

Is it possible to convert a code from one language to another using sequence to sequence programming. if not sequence to sequence, which algorithm can i use to do this?
3
votes
3answers
106 views

Predict output sequence one at a time with feedback

I would like to solve the following classification problem: Given an input sequence and zero or more initial terms of the true output, predict the next term in the output sequence. For example, my ...
2
votes
1answer
130 views

Is this a problem for a Seq2Seq model?

I'm struggling to find a tutorial/example which covers using an seq2seq model for sequential inputs other then text/translation. I have a multivariate dataset with n number of input variables each ...
0
votes
0answers
12 views

how to deal with varying output layer

i am trying to do Named Entity Recognition. So, for input, i am using entire text(converted to word level embedding as input) and out put as same length as input, but only the required entity will be ...
2
votes
0answers
61 views

Encoder-Decoder Sequence-to-Sequence Model for Translations in Both Directions

Is it possible to use a pre-trained sequence to sequence encoder-decoder model which translates an input text in source language to an output in target language to do an inverse translation? That is, ...
1
vote
0answers
26 views

On-the-fly seq2seq: starting translation before the input sequence ends

By now anybody even remotely familiar with RNNs has been exposed to the famous figure representing variants of such networks: However one type that is missing is one where there would be a partial ...
1
vote
0answers
75 views

Neural network outputting same result for all inputs

I'm building an encoder-decoder neural network in Keras for sequence generation. The specific task is to try and change the styles of the text. Both my encoder and decoders are LSTMs with latent ...
2
votes
1answer
117 views

Keras: Softmax output into embedding layer

I'm trying to build an encoder-decoder network in Keras to generate a sentence of a particular style. As my problem is unsupervised i.e. I don't have the ground truths for the generated sentences, I ...
1
vote
2answers
95 views

What are the benefits and tradeoffs of a 1D conv vs a multi-input seq2seq LSTM model?

I have 6 sequences, s1,..,s6. Using all sequences I want to predict a binary vector q = [0,0,0,1,1,1,0,0,0,1,1,1,...], which is a mask of the activity of the 6 sequences. I have looked at seq2seq ...
0
votes
0answers
91 views

Keras LSTM - How is it trained exactly?

I've been doing Deep Learning with Keras for a while now. As a starter to LSTMs, I thought of implementing a word-level prose generator with LSTMs in Keras. Basically, the LSTM network will learn ...
0
votes
1answer
50 views

Training Encoder-Decoder using Decoder Outputs

I am trying to build an encoder-decoder model for a text style transfer problem. The problem is I don't have parallel data between the two styles so I need to train the model in an unsupervised ...
0
votes
1answer
139 views

Do we really need `<unk>` tokens?

I am wondering, do we really need <unk> tokens? Why do we limit our vocabulary? Is it for speed? Accuracy? If we disable all limitations, what do you ...
0
votes
0answers
258 views

LSTM seq2seq results

I implmented the keras seq2seq model on timeseries data. I will not post a code because I think it is not a coding problem but the code can be found here (scroll down for the post) or look here The ...
1
vote
1answer
587 views

Very long sequence in neural networks

Beginner's question regarding sequences in neural networks: suppose I have classification problem that looks like: X = very long sequence of varying length. Y = class (assume for simplicity y=0/1). ...
1
vote
2answers
36 views

Group prediction

I have following sort of data coming every day: (0)(3,4,5)(6,9,1)(5,35,12,232) (1)(5,1,4)(6,2)(12,54,12,43)(8,23,65) (2)(6,7,2)(34,3) (3)(4323,23,12,4543) (4)(987,32,324,23,224,12,213,21)(1,2) (...
1
vote
1answer
76 views

How to pad real-valued sequences

I have several sequences of univariate real-valued time-series data. The sequences are of different lengths and right now I cannot batch them and feed them to a network. What is the correct procedure ...
2
votes
0answers
1k views

How/What to initialize the hidden states in RNN sequence-to-sequence models?

In an RNN sequence-to-sequence model, the encode input hidden states and the output's hidden states needs to be initialized before training. What values should we initialize them with? How should we ...
1
vote
0answers
337 views

How do attention mechanisms in RNNs learn weights for a variable length input

Attention mechanisms in RNNs are reasonably common to sequence to sequence models. I understand that the decoder learns a weight vector $\alpha$ which is applied as a weighted sum of the output ...
4
votes
0answers
480 views

Input for LSTM for financial time series directional prediction

I'm working on using an LSTM to predict the direction of the market for the next day. My question concerns the input for the LSTM. My data is a financial time series $x_1 \ldots x_t$ where each $x_i$...
4
votes
1answer
1k views

Why do we need to add START <s> + END </s> symbols when using Recurrent Neural Nets for Sequence-to-Sequence Models?

In the Sequence-to-Sequence models, we often see that the START (e.g. <s>) and END (e.g. </s>) symbols are added to ...