Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

493 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
16
votes
0answers
2k views

Understanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTT

What I'm trying to do What I am trying to do is predicting the next data-point $x_t$ for each point in the timeseries $[x_0, x_1, x_2,...,x_T]$ in the context of a date-stream in real-time, in theory ...
9
votes
2answers
1k views

Input for LSTM for financial time series directional prediction

I'm working on using an LSTM to predict the direction of the market for the next day. My question concerns the input for the LSTM. My data is a financial time series $x_1 \ldots x_t$ where each $x_i$...
8
votes
2answers
2k views

Forecasting Multiple (few hundreds) uni-variate time series with inflated zeros

I am a novice seeking help to gain experience in Data Science. Let us take a scenario where a big company would like to forecast its sales (a specific product) across different stores in different ...
6
votes
1answer
235 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
5
votes
1answer
132 views

LSTM Long Term Dependencies Keras

I am familiar with the LSTM unit (memory cell, forget gate, output gate etc) however I am struggling to see how this links to the LSTM implementation in Keras. In Keras the input data structure for X ...
5
votes
0answers
1k views

Multivariate, multistep forecasting with LSTM

I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" ...
4
votes
1answer
3k views

Autoencoders for the compression of time series

I am trying to use autoencoder (simple, convolutional, LSTM) to compress time series. Here are the models I tried. Simple autoencoder: ...
4
votes
1answer
872 views

Train LSTM model with multiple time series

I am predicting energy usage for a bedroom within a school residential building with date, temperature, and humidity as input features, using 7 time-steps and ...
4
votes
0answers
1k views

Using the Python Keras multi_gpu_model with LSTM / GRU to predict Timeseries data

I'm having an issue with python keras LSTM / GRU layers with multi_gpu_model for machine learning. When I use a single GPU, the predictions work correctly ...
3
votes
0answers
62 views

How is the input gate in the LSTM learn?

How is the input gate neural network trained what to remember by propagating the error rate from predicting the next word in the language model? How does it help it to learn if it remembered the right ...
3
votes
0answers
17 views

How respective gating functions are ensured in LSTM?

I'm studying the Hochreiter-Schmidhuber long-short term memory recurrent architecture. The overall idea, information flow and manipulation is clear, and it seemingly works, but what I cannot ...
3
votes
0answers
111 views

Help understanding the Tensorboard histogram names and meaning in an LSTM Model

Can someone please help me understand what the names and shapes of the following tensorboard histogram outputs mean about an LSTM model I coded? Thank you! I understand the terms in the names like ...
3
votes
0answers
226 views

Training an LSTM with different time steps and number of features

I want to use an LSTM using Keras to make course grade predictions. My dataset includes student transcripts, which consist of courses taken and their respective grades of students. For each course, I ...
3
votes
1answer
2k views

Running out of memory when training Keras LSTM model for binary classification on image sequences

I'm trying to come up with a Keras model based on LSTM layers that would do binary classification on image sequences. The input data has the following shape: ...
3
votes
0answers
55 views

Structuring a LSTM Layer

I'm trying to improve an NER Bert sequence tagger using LSTM layers in TensorFlow. I'm a bit unclear on the interface and how a LSTM layer should be set up. Currently, I'm taking in 3-5 sentences and ...
3
votes
0answers
262 views

Why embedding or rnn/lstm can not handle variable length sequence?

Pytorch embedding or lstm (I don't know about other dnn libraries) can not handle variable-length sequence by default. I am seeing various hacks to handle variable length. But my question is, why this ...
3
votes
0answers
243 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
3
votes
0answers
377 views

Grouping the Input Features for LSTM (keras)

When I have a input feature of 2-dimension (variable*feature), is it still good to flatten them into 1-dimension input ...
3
votes
0answers
244 views

convLSTM : how to structure input data

I have the following dataframe containing training data that I have been using to perform a regression task using CNN + FC : ...
3
votes
1answer
599 views
3
votes
1answer
488 views

Calculation of Output in LSTM Many-to-One Architecture

I'm new to Recurrent Neural Network but I want to train my data with LSTM but I'm having a trouble to understand LSTM Many-to-One architecture. Suppose the size of my data is ...
3
votes
2answers
80 views

Running an LSTM with Music Data

I'm working on a project for a class where I'm trying to create an algorithm that learns music and creates its own music. I'm having trouble on how to set up the data for it to be inputted into the ...
3
votes
1answer
59 views

Predicting t+1 from a set of sequences

Say I have have an experiment where I release a single rat into a maze and wait for it to reach the end. Say I also track this rat's position in the maze at various times. Let's do this $n$ times. Now,...
3
votes
0answers
348 views

For stateful LSTM, does sequence length matter?

With stateful LSTM the entire state is retained between both the sequences in the batch that is submitted, and even between separate batches until ...
3
votes
0answers
461 views

Neural Network Prediction regression task, output is a multiple factor of input with same peaks

When I missed some details please point this out. I made a simple sequential LSTM model for regression. The model loss is 3.2145e-06. The data is scaled between 0 and 1. I tried different variations ...
3
votes
0answers
49 views

Hochreiter LSTM (p. 4): Maximal values of logistic sigmoid derivative times weight

My questions follow the below page 4 excerpt from Hochreiter's LSTM paper: If $f_{l_{m}}$ is the logistic sigmoid function, then the maximal value of $f^\prime_{l_{m}}$ is 0.25. If $y^{l_{m-1}}$ ...
3
votes
0answers
878 views

TypeError: unsupported operand type(s) for %: 'int' and 'NoneType'(Stateful LSTM Keras)

So I have a trained LSTM model with which I am trying to predict future values. The model is stateful as seen below ...
2
votes
0answers
24 views

Models for Long-Term Time-Series Forecasting and Pattern Recognition

I'm trying to find a solution for long-term electricity hourly prices forecasting. Explaining simply, I have some data from 2018 - 2021 containing Demand, Renewable Generation, Hydropower Generation, ...
2
votes
0answers
23 views

Why the LSTM on Keras does not work correctly when it is necessary to predict several steps forward

I used AirPassenger Dataset. And based on several previous values(for examples 20) I want to predict several(3 or 5) steps in future. Like X -> y [10,20,30,....200]->[210,220,230] [20,30,40,.......
2
votes
1answer
20 views

LSTM returns the same results for different inputs

Hey everyone, I am working on a LSTM network in TensorFlow that predicts the values of the price-index of different product-categories in a month, based on those same values of the 12 months before. ...
2
votes
0answers
29 views

Why do RNNs share weight?

If weights are not shared the number of parameters will be extremely large and difficult to compute which I understand. I don't understand the argument that varying length inputs are taken care of by ...
2
votes
0answers
17 views

How to modify a Convolutional Neural Network architecture built for a univariate time series to multivariate time series?

I have built a CNN (in combination with a LSTM cell) that takes 1D time series-like data as an input and performs classification. I am obtaining a good performance, but the complete data has actually ...
2
votes
0answers
44 views

Determining Confidence Probabilities for Distribution

I have a LSTM neural network that predicts the number of items required in each room for the next time step. So for example for room A, B, C and D the model predicts A -> 3 items B -> 4 items C ...
2
votes
0answers
48 views

Time series forecasting. How use future values

I have a time series dataset containing hourly data from a few year, like below. Let's assume that I want to make prediction for the next 3 hours (2021-01-01 19:00, 2021-01-01 20:00, 2021-01-01 21:00)....
2
votes
0answers
17 views

Predicting millions of sparse timeseries using them to help each other

This is a very general problem faced by different types of businesses. Predict the future behavior of customers over time. Imagine that we have 1 million customers with their own features over time, ...
2
votes
0answers
210 views

Keras model with LSTM quantization aware training

I would like to run quantization aware training with a keras model which has an LSTM layer. However, just the LSTM layer seems to not be supported. Alan Chiao seems to suggest here that it is possible ...
2
votes
0answers
161 views

LSTM predicting same word repeatedly

I am sure that this has happened to you if you have trained an LSTM model. The LSTM model predicts the same word 2 or 3 times. Now, I am not saying that for every input it has the same output. I am ...
2
votes
0answers
22 views

LSTM variable period prediction

I'm trying to train a model to predict the final cost of a product being developed over a few months. I have historical data of similar products which will be used for training the model. Some of the ...
2
votes
0answers
147 views

Multiple features in LSTM

It's clear how LSTM works with 1 feature. But what happens if the number of features is > 1? According to the answer proposed here, Keras creates a computational graph that executes the sequence ...
2
votes
1answer
44 views

LSTM for time series forcasting

I manipulate the time series using the different structures of the neural networks in order to make a prediction, and I wonder if there is a way to choose the parameters of the networks intelligently? ...
2
votes
0answers
100 views

LSTM low training/validation error but really bad predictions

I'm building a LSTM model to create an automatic drums composer. I'm following this post: LSTM Metallica I've built my model and done all the enconding, I was able to emulate the behavior of the ...
2
votes
0answers
33 views

how to build lstm with functinal api?

I am having a time series prediction problem and the data set has 4 variables. My data set is like below: ...
2
votes
0answers
109 views

Streaming sequence detection (Binary Classification) LSTM/GRU

I am currently trying to implement a model which can detect a specific sequence according to the training data which looks like the following: ...
2
votes
0answers
47 views

LSTM Multivariate, structuring data

I'll jump right to the structure of the data, and then I'll ask the question(s): For a mass X ranging from 200 to 500 units, i have 100 seconds worth of 3 output_values. So, the first few rows of the ...
2
votes
1answer
145 views

How to implement LSTM using Doc2Vec vectors to get representation?

Hi all. I'm a newbie in ML. I read and found a paper about A Multi-Level Plagiarism Detection System Based on Deep Learning Algorithms and want to implement this model . But I can't find more about ...
2
votes
0answers
195 views

Data leakage in bidirectional LSTM timeseries data

Does it cause data leakage to train a bidirectional LSTM on data where a user can be a sample in the training data multiple times? Each row is a snapshot at a different point in time for a given ...
2
votes
0answers
130 views

Remove subwords from BERT output

I'm trying to build a multilingual WSD system with BERT on top as the embedding layer. In order to have better performances, after BERT finishes its job (and performs Transfer Learning), I need to ...
2
votes
1answer
117 views

What is the output of multivariate LSTM model?

I am currently trying to build an LSTM model by using multivariate inputs, but I don't understand what exact output I am predicting. I am currently using 5 features in the data as input data: ...
2
votes
2answers
2k views

How can I tune LSTM hyperparameters?

If anyone is there to answer these, that'll be great. I'm in the midst of a Final Year Project on LSTM. Currently, I’m stuck and confused over LSTM codes. There are 4 hyperparameters that I can play ...
2
votes
0answers
28 views

LONG SHORT-TERM MEMORY Hochreiter's paper BPTT

I was trying to read paper about LSTM, and I am stuck with mathematical problem. http://www.bioinf.jku.at/publications/older/2604.pdf page 4. see, |$f'_l$$_m$($net_l$$_m$)$w_l$$_m$ $_l$$_m$$_-$$_1$...

1
2 3 4 5
10