Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

Filter by
Sorted by
Tagged with
17
votes
1answer
4k views

Time Series prediction using LSTMs: Importance of making time series stationary

In this link on Stationarity and differencing, it has been mentioned that models like ARIMA require a stationarized time series for forecasting as it's statistical properties like mean, variance, ...
14
votes
2answers
4k views

Sliding window leads to overfitting in LSTM?

Will I overfit my LSTM if I train it via the sliding-window approach? Why do people not seem to use it for LSTMs? For a simplified example, assume that we have to predict the sequence of characters: ...
12
votes
1answer
618 views

So what's the catch with LSTM?

I am expanding my knowledge of the Keras package and I have been tooling with some of the available models. I have an NLP binary classification problem that I'm trying to solve and have been applying ...
12
votes
1answer
13k views

Multi-dimentional and multivariate Time-Series forecast (RNN/LSTM) Keras

I have been trying to understand how to represent and shape data to make a multidimentional and multivariate time series forecast using Keras (or TensorFlow) but I am still very unclear after reading ...
12
votes
3answers
6k views

Advantages of stacking LSTMs?

I'm wondering in what situations it is advantageous to stack LSTMs?
11
votes
4answers
7k views

Prediction interval around LSTM time series forecast

Is there a method to calculate the prediction interval (probability distribution) around a time series forecast from an LSTM (or other recurrent) neural network? Say, for example, I am predicting 10 ...
11
votes
2answers
7k views

How to implement “one-to-many” and “many-to-many” sequence prediction in Keras?

I struggle to interpret the Keras coding difference for one-to-many (e. g. classification of single images) and many-to-many (e. g. classification of image sequences) sequence labeling. I frequently ...
10
votes
2answers
4k views

Dropout on which layers of LSTM?

Using a multi-layer LSTM with dropout, is it advisable to put dropout on all hidden layers as well as the output Dense layers? In Hinton's paper (which proposed ...
10
votes
1answer
14k views

Keras LSTM with 1D time series

I'm learning how to use Keras and I've had reasonable success with my labelled dataset using the examples on Chollet's Deep Learning for Python. The data set is ~1000 Time Series with length 3125 with ...
10
votes
1answer
1k views

Using RNN (LSTM) for Gesture Recognition System

I'm trying to build a gesture recognition system for classifying ASL (American Sign Language) Gestures, so my input is supposed to be a sequence of frames either from a camera or a video file then it ...
9
votes
3answers
16k views

What does the output of model.predict function from Keras mean?

I have built a LSTM model to predict duplicate questions on the Quora official dataset. The test labels are 0 or 1. 1 indicates the question pair is duplicate. After building the model using ...
9
votes
3answers
180 views

Why do RNNs usually have fewer hidden layers than CNNs?

CNNs can have hundreds of hidden layers and since they are often used with image data, having many layers captures more complexity. However, as far as I have seen, RNNs usually have few layers e.g. ...
8
votes
3answers
16k views

What is LSTM, BiLSTM and when to use them?

I am very new to Deep learning and I am particularly interested in knowing what are LSTM and BiLSTM and when to use them (major application areas). Why are LSTM and BILSTM more popular than RNN? Can ...
8
votes
2answers
2k views

LSTM: How to deal with nonstationarity when predicting a time series

I want to do one-step-ahead predictions for time series with LSTM. To understand the algorithm, I built myself a toy example: A simple autocorrelated process. ...
7
votes
2answers
2k views

How to feed LSTM with different input array sizes?

If I like to write a LSTM network and feed it by different input array sizes, how is it possible? For example I want to get voice messages or text messages in a ...
7
votes
1answer
5k views

Multiple output for multi step ahead prediction using LSTM with keras

I am new to deep learning and LSTM (with keras). I am trying to solve a multi-step ahead time series prediction. I have 3 time series: A, B and C and I want to predict the values of C. I am training ...
6
votes
3answers
5k views

Validation loss is not decreasing

I am trying to train a LSTM model. Is this model suffering from overfitting? Here is train and validation loss graph:
6
votes
2answers
4k views

Batch Size of Stateful LSTM in keras

My Model is defined as below: defining the model ...
6
votes
2answers
176 views

Why real-world output of my classifier has similar label ratio to training data?

I trained a neural network on balanced dataset, and it has good accuracy ~85%. But in real world positives appear in about 10% of the cases or less. When I test network on set with real world ...
6
votes
2answers
2k views

How to train the same RNN over multiple series?

I have multiple separate time series and would like to train the same LSTM network on them. How to do in this situation? I can't just concatenate timeseries (along time), because I am afraid network ...
6
votes
1answer
9k views

What is the best method for classification of time series data? Should I use LSTM or a different method?

I am trying to classify raw accelerometer data x,y,z to its corresponding label. What is the best architecture for best results? Or, does anyone have any suggestions on LSTM architectures built on ...
6
votes
1answer
1k views

Binary classification of every time series step based on past and future values

I'm currently facing a Machine Learning problem and I've reached a point where I need some help to proceed. I have various time series of positional (x, ...
6
votes
1answer
914 views

Training with multi-series of different length with stateful LSTM

I'm training a stateful LSTM. My data is stored in a series of files, each file relates to a certain city. For each city I might have different amount of data, so City A I might have 4000 days, but ...
6
votes
1answer
4k views

How to use Embedding() with 3D tensor in Keras?

I have a list of stock price sequences with 20 timesteps each. That's a 2D array of shape (total_seq, 20). I can reshape it into (total_seq, 20, 1) for concatenation to other features. I also have ...
6
votes
2answers
5k views

When to use Stateful LSTM?

I'm trying to use LSTM on time-series data in order to generate future sequences that looks like the original sequences in term of values and progression direction. My approach is: train rnn to ...
6
votes
3answers
3k views

Questions about LSTM cells, units and inputs

I'm trying to learn how LSTM networks work, and even if I get the basics, the details of the internal structure is not clear for me. On this blog link, I found ...
6
votes
1answer
3k views

Is it necessary to perform rolling-window on LSTMs?

Let's say I have a set of n time-series with sequence length 8 [[a,b,c,d,e,f,g,h],[f,e,g,r,g,h,e,a],[a,e,r,a,k,e,l,i],...,[e,r,q,g,l,r,p,q]] And let's define the input that LSTM expects as a ...
6
votes
1answer
2k views

LSTM Feature selection process

We need to implement a time series problem by LSTM model. But, while implementing the same, the main challenge I am facing is feature selection issue. Because, our data-set contain 2300 observations ...
6
votes
1answer
99 views

Classifications as long-term memory and short-term memory in LSTM

How is the data classified as long-term memory and short-term memory? Is there some standards programmers set?
6
votes
0answers
164 views

How to arrange the dataset/images for CNN+LSTM

I am working on an image classification problem using Transfer Learning with Resnet50 as base model (in Keras) (For example Class A and Class B). There is a time factor involved in this ...
5
votes
2answers
727 views

Checking for stationarity in LSTM

An author in his blog checked for stationarity and removed them in a forecasting problem for using LSTM.I asked others and they said no need to check for it in LSTM.I read some articles and it looked ...
5
votes
3answers
2k views

LSTM for time series - which window size to use

I have a LSTM based network which inputs a n-sized sequence of length (n x 300) and outputs the next single step (1 x 300). The "raw" data consists of a few thousand semi-processed sequences of ...
5
votes
2answers
872 views

Anomaly detection using RNN LSTM

I'm trying to detect anomalies in an univariate time series. I trained a RNN LSTM and currently I get one-step-ahead predictions. Could someone explain if it's possible to output a confidence ...
5
votes
1answer
172 views

Suspected Exploding Gradient in Character Generator LSTM

I'm trying to create a neural network that can learn how to write text character by character from the book David Copperfield (via Project Gutenburg). It starts great, then forgets punctuation ...
5
votes
2answers
638 views

Kerns LSTM kernel

I am trying to understand how the weight matrix in an LSTM cell is used. An LSTM unit has several weight matrix: Wf, Wi, Wc, Wo like below: ( from http://colah....
5
votes
0answers
485 views

Understanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTT

What I'm trying to do What I am trying to do is predicting the next data-point $x_t$ for each point in the timeseries $[x_0, x_1, x_2,...,x_T]$ in the context of a date-stream in real-time, in theory ...
5
votes
1answer
607 views

Input for LSTM for financial time series directional prediction

I'm working on using an LSTM to predict the direction of the market for the next day. My question concerns the input for the LSTM. My data is a financial time series $x_1 \ldots x_t$ where each $x_i$...
4
votes
2answers
10k views

Value Error: Operands could not be broadcast together with shapes - LSTM

I am trying an LSTM model using tensorflow following this tutorial . I am having trouble understanding why am I getting an ...
4
votes
1answer
2k views

Relationship between batch size and the number of neurons in the input layer

Regarding LSTM neural networks, I am unable to understand the relationship between batch size, the number of neurons in the input layer and the number of "variables" or "columns" in the input. (...
4
votes
2answers
4k views

Time series forecasting with RNN(stateful LSTM) produces constant values

I have a time series daily data for about 6 years(1.8k data points). I am trying to forecast the next t+30 values, Train data independent matrix (X)=Sequences of previous 30 day values Train (Y)=The ...
4
votes
1answer
508 views

Why use two LSTM layers one after another?

In the example on the Keras site, seq2seq_translate.py on line 189, there is a LSTM layer after another (the first with return_sequences=True), but another example ...
4
votes
3answers
1k views

Handwritting Recognition moving from character level to word level

Given the experience on MIST, I try this problem as a character level. I have a handwritten text and I want to "OCR" it. Even though I made progresses with openCV (on the image pre-processing, ...
4
votes
1answer
368 views

How to implement LSTM with Spark?

I would like to build an LSTM network for text classification with PySpark, but I don't find any library or function about it. ...
4
votes
1answer
36 views

Why my results have time delay when I use LSTM?

I am trying to fit and test LSTM on a numeric series(like stock prices). But it seems that I always get a lag in predicted graph(Blue) with respect to real graph(red). Does anyone know why this ...
4
votes
2answers
481 views

Understanding LSTM input shape for keras

I am learning about the LSTM network. The input needs to be 3D. So I have a CSV file which has 9999 data with one feature only. So it is only one file. So usually it is ...
4
votes
1answer
604 views

what actually word embedding dimensions values represent?

I am learning word2vec and word embedding , I have downloaded GloVe pre-trained word embedding (shape 40,000 x 50) and using this function to extract information from that: ...
4
votes
1answer
132 views

On the choice of LSTM input/output dimension for a spatio-temporal problem

I am using LSTM neural networks from (R)Keras for a matter of spatio-temporal interpolation. I manage to get the network to output predictions but the results are not outstanding (very little ...
4
votes
1answer
78 views

LSTM training/prediction with no starting sequence

ML newbie here. As an exercise, I'm trying to build a character based language model based on a simple 1 layer LSTM. Based on what I've learned about LSTMs, a common usage is to take in a sequence of ...
4
votes
2answers
466 views

Understanding output of LSTM for regression

Please see the update, below. I am working with embeddings and wanted to see how feasible it is to predict some scores attached to some sequences of words. The details of the scores are not important....
4
votes
1answer
121 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...