Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

420 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
8
votes
0answers
1k views

Understanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTT

What I'm trying to do What I am trying to do is predicting the next data-point $x_t$ for each point in the timeseries $[x_0, x_1, x_2,...,x_T]$ in the context of a date-stream in real-time, in theory ...
6
votes
1answer
727 views

Input for LSTM for financial time series directional prediction

I'm working on using an LSTM to predict the direction of the market for the next day. My question concerns the input for the LSTM. My data is a financial time series $x_1 \ldots x_t$ where each $x_i$...
4
votes
1answer
609 views

Train LSTM model with multiple time series

I am predicting energy usage for a bedroom within a school residential building with date, temperature, and humidity as input features, using 7 time-steps and ...
4
votes
0answers
109 views

LSTM Long Term Dependencies Keras

I am familiar with the LSTM unit (memory cell, forget gate, output gate etc) however I am struggling to see how this links to the LSTM implementation in Keras. In Keras the input data structure for X ...
4
votes
1answer
160 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
4
votes
1answer
1k views

How to draw a simple LSTM network

I'm new to deep learning, I am learning LSTM for my PhD work. This is a simple LSTM network for sequence classification. This code is from MATLAB tutorial: ...
4
votes
0answers
3k views

Kalman filter for time series prediction

I have the information about the behaviour of 400 users across period of 1 months (30 days). Across those 30 days I measure 4 different information (let's call it A,B,C and D), hence I have a total of ...
3
votes
0answers
30 views

N-grams for RNNs

Given a word $w_{n}$ a statistical model such a Markov chain using n-grams predicts the subsequent word $w_{n+1}$. The prediction is by no means random. How is this translated into a neural model? I ...
3
votes
1answer
667 views

Running out of memory when training Keras LSTM model for binary classification on image sequences

I'm trying to come up with a Keras model based on LSTM layers that would do binary classification on image sequences. The input data has the following shape: ...
3
votes
0answers
211 views

Why embedding or rnn/lstm can not handle variable length sequence?

Pytorch embedding or lstm (I don't know about other dnn libraries) can not handle variable-length sequence by default. I am seeing various hacks to handle variable length. But my question is, why this ...
3
votes
0answers
178 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
3
votes
1answer
422 views

Do I need to engineer lagged features when creating an LSTM for time series forecasting?

Long short-term memory networks are fairly complicated and I haven't completely wrapped my head around them. It seems to me like the big gain in LSTMs for time series forecasting is the lacking ...
3
votes
0answers
216 views

convLSTM : how to structure input data

I have the following dataframe containing training data that I have been using to perform a regression task using CNN + FC : ...
3
votes
2answers
388 views

What is the difference between “Adding more LSTM layers” or “Adding more units on existence layers”?

What is the difference between adding more LSTM layers and just increasing the units of existing layers? Which one is preferred and in which situation?
3
votes
0answers
1k views

Using the Python Keras multi_gpu_model with LSTM / GRU to predict Timeseries data

I'm having an issue with python keras LSTM / GRU layers with multi_gpu_model for machine learning. When I use a single GPU, the predictions work correctly ...
3
votes
2answers
549 views

Beyond one-hot encoding for LSTM model in Keras

I have an LSTM model in Keras for categorical classification (20 possible categories). In many cases, my data can fit multiple categories. Obviously, my current model uses one-hot encoding and fits ...
3
votes
1answer
43 views

Predicting t+1 from a set of sequences

Say I have have an experiment where I release a single rat into a maze and wait for it to reach the end. Say I also track this rat's position in the maze at various times. Let's do this $n$ times. Now,...
3
votes
0answers
949 views

Multivariate, multistep forecasting with LSTM

I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" ...
3
votes
0answers
296 views

For stateful LSTM, does sequence length matter?

With stateful LSTM the entire state is retained between both the sequences in the batch that is submitted, and even between separate batches until ...
3
votes
0answers
394 views

Neural Network Prediction regression task, output is a multiple factor of input with same peaks

When I missed some details please point this out. I made a simple sequential LSTM model for regression. The model loss is 3.2145e-06. The data is scaled between 0 and 1. I tried different variations ...
3
votes
0answers
48 views

Hochreiter LSTM (p. 4): Maximal values of logistic sigmoid derivative times weight

My questions follow the below page 4 excerpt from Hochreiter's LSTM paper: If $f_{l_{m}}$ is the logistic sigmoid function, then the maximal value of $f^\prime_{l_{m}}$ is 0.25. If $y^{l_{m-1}}$ ...
3
votes
0answers
656 views

TypeError: unsupported operand type(s) for %: 'int' and 'NoneType'(Stateful LSTM Keras)

So I have a trained LSTM model with which I am trying to predict future values. The model is stateful as seen below ...
2
votes
0answers
40 views

LSTM Multivariate, structuring data

I'll jump right to the structure of the data, and then I'll ask the question(s): For a mass X ranging from 200 to 500 units, i have 100 seconds worth of 3 output_values. So, the first few rows of the ...
2
votes
0answers
45 views

Remove subwords from BERT output

I'm trying to build a multilingual WSD system with BERT on top as the embedding layer. In order to have better performances, after BERT finishes its job (and performs Transfer Learning), I need to ...
2
votes
0answers
25 views

LONG SHORT-TERM MEMORY Hochreiter's paper BPTT

I was trying to read paper about LSTM, and I am stuck with mathematical problem. http://www.bioinf.jku.at/publications/older/2604.pdf page 4. see, |$f'_l$$_m$($net_l$$_m$)$w_l$$_m$ $_l$$_m$$_-$$_1$...
2
votes
0answers
58 views

Help understanding the Tensorboard histogram names and meaning in an LSTM Model

Can someone please help me understand what the names and shapes of the following tensorboard histogram outputs mean about an LSTM model I coded? Thank you! I understand the terms in the names like ...
2
votes
0answers
49 views

Dense? or TimeDistributedDense? after LSTM layer in Keras

Dense and TimeDistributedDense, which one is suitable after LSTM layer in Keras? For example, ...
2
votes
0answers
39 views

Masking seems not working for missing values problem in LSTM

I am trying to use LSTM to predict time series in keras. My input data shape is (1000,6,1)(samples,timesteps,features). There is some missing data in different timesteps. For example,[2,1,1]=NaN,[3,4,...
2
votes
0answers
58 views

Setting up RNN in TensorFlow for time series forecast with variable input series lengths

I am building a model with keras for time series prediction. The structure of the problem is as follows: The input is a time series of 5 numeric features The ...
2
votes
0answers
104 views

Using LSTM to forecast vehicle position - multivariate time series - Matlab

I am trying to train an LSTM model on Matlab to forecast the position of a vehicle when driving around a roundabout. My main concern right now is that my dataset consists of 4 features (X position, Y ...
2
votes
0answers
22 views

Wiggle in the initial part of an LSTM prediction

I working on using LSTMs and GRUs to make time series predictions. For the most part the predictions are pretty good. However, there seems to be a wiggle (or initial up-then-down) before the ...
2
votes
0answers
55 views

Training an LSTM with different time steps and number of features

I want to use an LSTM using Keras to make course grade predictions. My dataset includes student transcripts, which consist of courses taken and their respective grades of students. For each course, I ...
2
votes
1answer
1k views

Autoencoders for the compression of time series

I am trying to use autoencoder (simple, convolutional, LSTM) to compress time series. Here are the models I tried. Simple autoencoder: ...
2
votes
0answers
56 views

How can RNN handle variable sized inputs?

I came across this answer which is specific to Keras. But my question is at concept level. I am getting confused, How can RNN handle variable size inputs? here Let us suppose we want to do a ...
2
votes
2answers
330 views

Generate new sentences based on keywords

For example, for a domain specific neural network in Fashion, with the Keywords light, dress, orange, cotton. It could output: This gorgeous orange summer dress is great for wearing on sunny camping ...
2
votes
0answers
41 views

NLP based Data Preprocessing Method to Improve Disease Name Prediction Using CRF and Word Embedding

I built a model( using CRF along bi lstm) to Predict New Disease Name/Entities from medical text data but the problem is Disease name appears only 5,6 times in 1 text file but on average text file ...
2
votes
0answers
16 views

Simmultainiously calculting loss from target interdependend metric

Is there a way to incorporate multiple targets into one loss? Currently, I work with the Sequential() API, I guess this won't be sufficient.... I work with area predictions as targets. Each sample ...
2
votes
0answers
44 views

Structuring a LSTM Layer

I'm trying to improve an NER Bert sequence tagger using LSTM layers in TensorFlow. I'm a bit unclear on the interface and how a LSTM layer should be set up. Currently, I'm taking in 3-5 sentences and ...
2
votes
0answers
22 views

Why might an LSTM be capable of predicting an ARMA signal but not a linear combination of ARMA signals?

I have an LSTM network and am testing it on some dummy ARMA signals. I'm trying to predict the signal 5 time steps into the future. The network is capable of outperforming Naive (persistence) when ...
2
votes
0answers
20 views

hidden state of each sequence of mini-batch

I am new to Pytorch and trying to implement a lstm character level seq2seq model. What I am trying to do is: Each sequence is a list of the characters of a particular word and several words will ...
2
votes
0answers
46 views

What receives a LSTM neuron?

I'm confused about input data on LSTM neurons. I know that exist almost two form to give data to a recurrent neural network. I want to understand with a example ...
2
votes
1answer
217 views

What are the equations involved in calculation of the parameters of embedding layer?

I'm trying to perform sentiment analysis on some data using keras.I'm using embedding layer and then LSTM. I know that embedding layer decreases the sparsity of the one hot encodings of the words and ...
2
votes
0answers
181 views

How smaller does the input data get reduced in a LSTM autoencoder

Question In a LSTM autoencder, how smaller does my input data(59 features) get reduced in a latent vector, which is usually located in the middle between an encoder and a decoder? Why did the ...
2
votes
0answers
363 views

Measuring uncertainty in an LSTM network using dropout in keras/tensorflow

I've created a simple LSTM network for testing ...
2
votes
1answer
880 views

ValueError: Cannot convert a partially known TensorShape to a Tensor: (?, 256)

I'm working on a sequence to sequence approach using LSTM and a VAE with an attention mechanism. ...
2
votes
1answer
724 views

LSTM: Converting to Bayesian Deep Neural Network

Starting from Yarin Gal's research paper on using Dropout as a Bayesian Approximation (https://arxiv.org/pdf/1506.02142.pdf), I am trying to apply this concept to my Sequence Prediction model. My ...
2
votes
0answers
176 views

Grouping the Input Features for LSTM (keras)

When I have a input feature of 2-dimension (variable*feature), is it still good to flatten them into 1-dimension input ...
2
votes
0answers
563 views

LSTM not converging

I am sorry if this questions is basic but I am quite new to NN in general. I am trying to build an LSTM to predict certain properties of a light curve (the output is 0 or 1). I build it in pytorch. ...
2
votes
1answer
149 views

How to add previous predictions for new predictions in LSTM?

I am trying to train a model on a big data sequence like this [0.2 0.1 0.1 ..... 0.4 0.8] . I create X vectors with length 60 for inputs and Y scaler numbers as ...
2
votes
1answer
428 views

1
2 3 4 5
9