Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

432 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
12
votes
0answers
1k views

Understanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTT

What I'm trying to do What I am trying to do is predicting the next data-point $x_t$ for each point in the timeseries $[x_0, x_1, x_2,...,x_T]$ in the context of a date-stream in real-time, in theory ...
10
votes
1answer
784 views

Input for LSTM for financial time series directional prediction

I'm working on using an LSTM to predict the direction of the market for the next day. My question concerns the input for the LSTM. My data is a financial time series $x_1 \ldots x_t$ where each $x_i$...
7
votes
2answers
1k views

Forecasting Multiple (few hundreds) uni-variate time series with inflated zeros

Hello Practitioners, Being a newbie seeking help to gain experience in Data Science. Lets take a scenario where a big company wants to forecast its sales (a specific product) across different stores ...
5
votes
1answer
46 views

N-grams for RNNs

Given a word $w_{n}$ a statistical model such a Markov chain using n-grams predicts the subsequent word $w_{n+1}$. The prediction is by no means random. How is this translated into a neural model? I ...
5
votes
1answer
166 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
4
votes
1answer
657 views

Train LSTM model with multiple time series

I am predicting energy usage for a bedroom within a school residential building with date, temperature, and humidity as input features, using 7 time-steps and ...
4
votes
0answers
110 views

LSTM Long Term Dependencies Keras

I am familiar with the LSTM unit (memory cell, forget gate, output gate etc) however I am struggling to see how this links to the LSTM implementation in Keras. In Keras the input data structure for X ...
4
votes
0answers
1k views

Using the Python Keras multi_gpu_model with LSTM / GRU to predict Timeseries data

I'm having an issue with python keras LSTM / GRU layers with multi_gpu_model for machine learning. When I use a single GPU, the predictions work correctly ...
4
votes
0answers
3k views

Kalman filter for time series prediction

I have the information about the behaviour of 400 users across period of 1 months (30 days). Across those 30 days I measure 4 different information (let's call it A,B,C and D), hence I have a total of ...
3
votes
0answers
40 views

How is the input gate in the LSTM learn?

How is the input gate neural network trained what to remember by propagating the error rate from predicting the next word in the language model? How does it help it to learn if it remembered the right ...
3
votes
1answer
20 views

LSTM for time series forcasting

I manipulate the time series using the different structures of the neural networks in order to make a prediction, and I wonder if there is a way to choose the parameters of the networks intelligently? ...
3
votes
0answers
16 views

How respective gating functions are ensured in LSTM?

I'm studying the Hochreiter-Schmidhuber long-short term memory recurrent architecture. The overall idea, information flow and manipulation is clear, and it seemingly works, but what I cannot ...
3
votes
1answer
829 views

Running out of memory when training Keras LSTM model for binary classification on image sequences

I'm trying to come up with a Keras model based on LSTM layers that would do binary classification on image sequences. The input data has the following shape: ...
3
votes
1answer
2k views

Autoencoders for the compression of time series

I am trying to use autoencoder (simple, convolutional, LSTM) to compress time series. Here are the models I tried. Simple autoencoder: ...
3
votes
2answers
1k views

Generate new sentences based on keywords

For example, for a domain specific neural network in Fashion, with the Keywords light, dress, orange, cotton. It could output: This gorgeous orange summer dress is great for wearing on sunny camping ...
3
votes
0answers
47 views

Structuring a LSTM Layer

I'm trying to improve an NER Bert sequence tagger using LSTM layers in TensorFlow. I'm a bit unclear on the interface and how a LSTM layer should be set up. Currently, I'm taking in 3-5 sentences and ...
3
votes
0answers
237 views

Why embedding or rnn/lstm can not handle variable length sequence?

Pytorch embedding or lstm (I don't know about other dnn libraries) can not handle variable-length sequence by default. I am seeing various hacks to handle variable length. But my question is, why this ...
3
votes
0answers
199 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
3
votes
0answers
226 views

Grouping the Input Features for LSTM (keras)

When I have a input feature of 2-dimension (variable*feature), is it still good to flatten them into 1-dimension input ...
3
votes
0answers
222 views

convLSTM : how to structure input data

I have the following dataframe containing training data that I have been using to perform a regression task using CNN + FC : ...
3
votes
2answers
454 views

What is the difference between “Adding more LSTM layers” or “Adding more units on existence layers”?

What is the difference between adding more LSTM layers and just increasing the units of existing layers? Which one is preferred and in which situation?
3
votes
2answers
572 views

Beyond one-hot encoding for LSTM model in Keras

I have an LSTM model in Keras for categorical classification (20 possible categories). In many cases, my data can fit multiple categories. Obviously, my current model uses one-hot encoding and fits ...
3
votes
1answer
49 views

Predicting t+1 from a set of sequences

Say I have have an experiment where I release a single rat into a maze and wait for it to reach the end. Say I also track this rat's position in the maze at various times. Let's do this $n$ times. Now,...
3
votes
0answers
991 views

Multivariate, multistep forecasting with LSTM

I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" ...
3
votes
2answers
706 views

Is this a problem for a Seq2Seq model?

I'm struggling to find a tutorial/example which covers using an seq2seq model for sequential inputs other then text/translation. I have a multivariate dataset with n number of input variables each ...
3
votes
0answers
302 views

For stateful LSTM, does sequence length matter?

With stateful LSTM the entire state is retained between both the sequences in the batch that is submitted, and even between separate batches until ...
3
votes
0answers
409 views

Neural Network Prediction regression task, output is a multiple factor of input with same peaks

When I missed some details please point this out. I made a simple sequential LSTM model for regression. The model loss is 3.2145e-06. The data is scaled between 0 and 1. I tried different variations ...
3
votes
0answers
49 views

Hochreiter LSTM (p. 4): Maximal values of logistic sigmoid derivative times weight

My questions follow the below page 4 excerpt from Hochreiter's LSTM paper: If $f_{l_{m}}$ is the logistic sigmoid function, then the maximal value of $f^\prime_{l_{m}}$ is 0.25. If $y^{l_{m-1}}$ ...
3
votes
0answers
706 views

TypeError: unsupported operand type(s) for %: 'int' and 'NoneType'(Stateful LSTM Keras)

So I have a trained LSTM model with which I am trying to predict future values. The model is stateful as seen below ...
2
votes
0answers
22 views

Difference between cell state and hidden state

What confuses me the most about LSTM cells are the cell state and hidden state. How are these both different from each other? What information do they both carry?
2
votes
0answers
17 views

LSTM variable period prediction

I'm trying to train a model to predict the final cost of a product being developed over a few months. I have historical data of similar products which will be used for training the model. Some of the ...
2
votes
0answers
57 views

Multiple features in LSTM

It's clear how LSTM works with 1 feature. But what happens if the number of features is > 1? According to the answer proposed here, Keras creates a computational graph that executes the sequence ...
2
votes
0answers
17 views

LSTM low training/validation error but really bad predictions

I'm building a LSTM model to create an automatic drums composer. I'm following this post: LSTM Metallica I've built my model and done all the enconding, I was able to emulate the behavior of the ...
2
votes
0answers
15 views

how to build lstm with functinal api?

I am having a time series prediction problem and the data set has 4 variables. My data set is like below: ...
2
votes
0answers
39 views

Streaming sequence detection (Binary Classification) LSTM/GRU

I am currently trying to implement a model which can detect a specific sequence according to the training data which looks like the following: ...
2
votes
0answers
43 views

LSTM Multivariate, structuring data

I'll jump right to the structure of the data, and then I'll ask the question(s): For a mass X ranging from 200 to 500 units, i have 100 seconds worth of 3 output_values. So, the first few rows of the ...
2
votes
0answers
57 views

Remove subwords from BERT output

I'm trying to build a multilingual WSD system with BERT on top as the embedding layer. In order to have better performances, after BERT finishes its job (and performs Transfer Learning), I need to ...
2
votes
0answers
26 views

LONG SHORT-TERM MEMORY Hochreiter's paper BPTT

I was trying to read paper about LSTM, and I am stuck with mathematical problem. http://www.bioinf.jku.at/publications/older/2604.pdf page 4. see, |$f'_l$$_m$($net_l$$_m$)$w_l$$_m$ $_l$$_m$$_-$$_1$...
2
votes
0answers
70 views

Help understanding the Tensorboard histogram names and meaning in an LSTM Model

Can someone please help me understand what the names and shapes of the following tensorboard histogram outputs mean about an LSTM model I coded? Thank you! I understand the terms in the names like ...
2
votes
0answers
67 views

Dense? or TimeDistributedDense? after LSTM layer in Keras

Dense and TimeDistributedDense, which one is suitable after LSTM layer in Keras? For example, ...
2
votes
0answers
66 views

Masking seems not working for missing values problem in LSTM

I am trying to use LSTM to predict time series in keras. My input data shape is (1000,6,1)(samples,timesteps,features). There is some missing data in different timesteps. For example,[2,1,1]=NaN,[3,4,...
2
votes
0answers
66 views

Setting up RNN in TensorFlow for time series forecast with variable input series lengths

I am building a model with keras for time series prediction. The structure of the problem is as follows: The input is a time series of 5 numeric features The ...
2
votes
1answer
96 views

Impact of varying sequence length in ensemble GRU model

I am using ensemble gru for my project and keeping different cell sizes for different models !For example, first gru model is of size 16 and the second is of 8 and 4 for the third model. The model is ...
2
votes
0answers
128 views

Using LSTM to forecast vehicle position - multivariate time series - Matlab

I am trying to train an LSTM model on Matlab to forecast the position of a vehicle when driving around a roundabout. My main concern right now is that my dataset consists of 4 features (X position, Y ...
2
votes
0answers
24 views

Wiggle in the initial part of an LSTM prediction

I working on using LSTMs and GRUs to make time series predictions. For the most part the predictions are pretty good. However, there seems to be a wiggle (or initial up-then-down) before the ...
2
votes
0answers
73 views

Training an LSTM with different time steps and number of features

I want to use an LSTM using Keras to make course grade predictions. My dataset includes student transcripts, which consist of courses taken and their respective grades of students. For each course, I ...
2
votes
0answers
77 views

How can RNN handle variable sized inputs?

I came across this answer which is specific to Keras. But my question is at concept level. I am getting confused, How can RNN handle variable size inputs? here Let us suppose we want to do a ...
2
votes
0answers
50 views

NLP based Data Preprocessing Method to Improve Disease Name Prediction Using CRF and Word Embedding

I built a model( using CRF along bi lstm) to Predict New Disease Name/Entities from medical text data but the problem is Disease name appears only 5,6 times in 1 text file but on average text file ...
2
votes
0answers
17 views

Simmultainiously calculting loss from target interdependend metric

Is there a way to incorporate multiple targets into one loss? Currently, I work with the Sequential() API, I guess this won't be sufficient.... I work with area predictions as targets. Each sample ...
2
votes
0answers
25 views

Why might an LSTM be capable of predicting an ARMA signal but not a linear combination of ARMA signals?

I have an LSTM network and am testing it on some dummy ARMA signals. I'm trying to predict the signal 5 time steps into the future. The network is capable of outperforming Naive (persistence) when ...

1
2 3 4 5
9