Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

Filter by
Sorted by
Tagged with
7
votes
2answers
3k views

How to feed LSTM with different input array sizes?

If I like to write a LSTM network and feed it by different input array sizes, how is it possible? For example I want to get voice messages or text messages in a ...
12
votes
1answer
14k views

Multi-dimentional and multivariate Time-Series forecast (RNN/LSTM) Keras

I have been trying to understand how to represent and shape data to make a multidimentional and multivariate time series forecast using Keras (or TensorFlow) but I am still very unclear after reading ...
10
votes
1answer
14k views

Keras LSTM with 1D time series

I'm learning how to use Keras and I've had reasonable success with my labelled dataset using the examples on Chollet's Deep Learning for Python. The data set is ~1000 Time Series with length 3125 with ...
14
votes
2answers
4k views

Sliding window leads to overfitting in LSTM?

Will I overfit my LSTM if I train it via the sliding-window approach? Why do people not seem to use it for LSTMs? For a simplified example, assume that we have to predict the sequence of characters: ...
6
votes
1answer
1k views

Binary classification of every time series step based on past and future values

I'm currently facing a Machine Learning problem and I've reached a point where I need some help to proceed. I have various time series of positional (x, ...
6
votes
3answers
7k views

Validation loss is not decreasing

I am trying to train a LSTM model. Is this model suffering from overfitting? Here is train and validation loss graph:
1
vote
2answers
554 views

loss/val_loss are decreasing but accuracies are the same in LSTM!

I am trying to train a LSTM model, but the problem is that the loss and val_loss are decreasing from 12 and 5 to less than 0.01, but the training set ...
11
votes
2answers
8k views

How to implement “one-to-many” and “many-to-many” sequence prediction in Keras?

I struggle to interpret the Keras coding difference for one-to-many (e. g. classification of single images) and many-to-many (e. g. classification of image sequences) sequence labeling. I frequently ...
8
votes
2answers
2k views

LSTM: How to deal with nonstationarity when predicting a time series

I want to do one-step-ahead predictions for time series with LSTM. To understand the algorithm, I built myself a toy example: A simple autocorrelated process. ...
11
votes
4answers
7k views

Prediction interval around LSTM time series forecast

Is there a method to calculate the prediction interval (probability distribution) around a time series forecast from an LSTM (or other recurrent) neural network? Say, for example, I am predicting 10 ...
7
votes
1answer
10k views

What is the best method for classification of time series data? Should I use LSTM or a different method?

I am trying to classify raw accelerometer data x,y,z to its corresponding label. What is the best architecture for best results? Or, does anyone have any suggestions on LSTM architectures built on ...
3
votes
1answer
392 views

RNNs for time series prediction - what configurations would make sense

My question here is mostly about general-intuition logic: when using a RNN (LSTM) for predicting a time series, and you have the goal of, for example, predicting at ...
3
votes
1answer
3k views

The model of LSTM with more than one unit

In stacked LSTM, for example: 2 LSTM layers, LSTM_1 in order to pass the output of every time step to LSTM_2, so it needs to return hidden state value in every time step, like the architecture I drew ...
2
votes
1answer
111 views

LSTM querying approach

I've just realized my prediction approach for LSTM might not be correct. I am trying to predict character by character, by reading over the book. The way I've approached the problem is as follows: <...
2
votes
0answers
441 views

Recommended model for univariate or multivariate multistep ahead time series forecasting

I have a dataset consisting of recurring and non-recurring expense transactions from bank accounts, as well as other features describing the bank account and each transation. I aggregate these ...
7
votes
2answers
5k views

Batch Size of Stateful LSTM in keras

My Model is defined as below: defining the model ...
3
votes
2answers
3k views

What is the job of “RepeatVector” and “TimeDistributed”?

I read about them in Keras documentation and other websites, but I couldn't exactly understand what exactly they do and how should we use them in designing ...
3
votes
1answer
1k views

Multivariate Time-Series forecasting using LSTM

I have a dataset of hourly measures of pollution('Sample_Measurement) and weather condition. If I want to predict the pollution level of the current hour using the weather and pollution data of the ...
2
votes
0answers
174 views

Gradient derivation reference for Phased LSTM [closed]

I've worked-through the back-propagation for the Phased LSTM (Daniel Neil, Michael Pfeiffer, and Shih-Chii Liu 2016) and would like to show the notes. It was a relatively difficult task, so I post it ...
3
votes
1answer
347 views

Stateful LSTM : Using different training window

Would it make sense for stateful LSTM (or LSTM in general) if in one epoch I feed [0-9],[10-19],[20-29],[30-39]...[990-999] (with corresponding labels/Y data) from my dataset. When I've presented all ...
3
votes
1answer
74 views

Is it a red flag that increasing the number of parameters makes the model less able to overfit small amounts of data?

I'm training a deep network (CNN-LSTM-CRF) for Named Entity Recognition. Is there a reason that increasing the number of parameters would make the network less able to overfit a small training set (~...
2
votes
1answer
2k views

How to implement LSTM using Doc2Vec vectors?

I would like to build a ANN for text classification, which has an LSTM layer, and using weights obtained via a Doc2Vec model ...
2
votes
1answer
169 views

How to design batches in a stateful RNN

I am using TF Eager to train a stateful RNN (GRU). I have several variable length time sequences about 1 minute long which I split into windows of length 1s. In TF Eager, like in Keras, if ...
2
votes
1answer
111 views

Help framing a sequence prediction problem

I've found lots of tutorial/examples that focus on sequence prediction, which use previous time steps of the input variable(s) in order to create a forecast e.g. predict stock market price based on ...
2
votes
2answers
1k views

Softmax classifier never allows for 100% probability in LSTM?

When working with LSTM I am using a softmax classifier and a one-hot encoded vector approach. The softmax looks like this: $$S(h_i) = \frac{e^{h_i}}{\sum e^{h_{total}}}$$ notice, LSTM's result is a $...
2
votes
0answers
794 views

Multivariate, multistep forecasting with LSTM

I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" ...
2
votes
1answer
609 views

How to reshape data for LSTM training in multivariate sequence prediction

I want to build an LSTM model for customer behaviour. It's the first time for me working on a timeseries, so some concepts are not clear to me at all. My prediction problem is multidimensional, ...
1
vote
1answer
142 views

How to optimally train deep learning model using output as new input

I'm trying to train a network to predict the future. My current setup uses 5 time steps as inputs from the past, each consisting of 10 features, resulting in a [5, 10] input matrix (initially ...
1
vote
1answer
386 views

How to design a many-to-many LSTM?

I have an input array of shape (1000,20, 4) and output(labels) of shape (1000,25,1). But don't know how to use Keras LSTM library to build a sequential model for this! Can someone help me design a ...
1
vote
1answer
375 views

How to set input for proper fit with lstm?

My input training and test dataset is the following size: ...
1
vote
1answer
78 views

How to add previous predictions for new predictions in LSTM?

I am trying to train a model on a big data sequence like this [0.2 0.1 0.1 ..... 0.4 0.8] . I create X vectors with length 60 for inputs and Y scaler numbers as ...
1
vote
1answer
225 views

Mini batch size and reset states

3 is a big file, but I would like to reset the state after mini_batch_size of 50. ...
0
votes
0answers
29 views

How to use TimeDistributed fo CNN+LSTM?

I am trying to classify 6 classes time-frequency domain signal (STFT spectrogram) with a size of 3601x217 pixels. Assume that for each classes have 70 training samples, 20 validation samples, and 10 ...
0
votes
1answer
93 views

Sequences of time series data with only 1 output classification

So I'm facing a problem where I have a sequence (30h of data with 10sec intervals) and which is labeled to a class (3 different classes) for the whole sequence. I'm used to work with time series who ...
0
votes
0answers
66 views

How to shape input for multi-site multivariate time-series forecasting in LSTM?

Can anyone help me with how to shape the input for multi-site multivariate time-series? My dataset is something like: One csv file for each time-step: Each file contains 5 parameters (x,y, u-velocity,...