Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

Filter by
Sorted by
Tagged with
0
votes
1answer
56 views

Keras RNN (batch_size

I created RNN model for text classification with LSTM layer, but when I put the batch_size in the fit method, my model trained on the whole batch instead of just the mini batch _size. This also ...
1
vote
2answers
39 views

h in LSTM increasing in size?

So I was reading about the LSTM architecture and I was having trouble understanding a certain aspect of it. This article mentions the step in question near the bottom of the page. Here is the image ...
2
votes
2answers
96 views

What is the role of $W_{ax}, W_{aa}, W_{ay}$ in forward propagation in RNN? Are they hyperparameters? Why are they needed?

In RNN introduction in Coursera sequence model course, the following formula for forward propagation in RNN was introduced. What exactly is the role of $W_{ax}, W_{aa}, W_{ay}$? What do they do? In ...
1
vote
1answer
40 views

Using LSTMs for continous learning and predicting

I'm trying to develop a model to predict a commodity price movement direction based on previous observations. The model should learn common technical analysis patterns, e.g. head and shoulders. So, I ...
9
votes
1answer
3k views

Activation function between LSTM layers

I'm aware the LSTM cell uses both sigmoid and tanh activation functions internally, however when creating a stacked LSTM architecture does it make sense to pass their outputs through an activation ...
2
votes
0answers
53 views

Dense? or TimeDistributedDense? after LSTM layer in Keras

Dense and TimeDistributedDense, which one is suitable after LSTM layer in Keras? For example, ...
2
votes
0answers
43 views

Masking seems not working for missing values problem in LSTM

I am trying to use LSTM to predict time series in keras. My input data shape is (1000,6,1)(samples,timesteps,features). There is some missing data in different timesteps. For example,[2,1,1]=NaN,[3,4,...
3
votes
1answer
340 views

TensorFlow / Keras: What is stateful = True in LSTM layers?

Could you elaborate on this argument? I found the brief explanation from the docs unsatisfying: stateful: Boolean (default False). If True, the last state for each sample at index i in a batch will ...
2
votes
1answer
66 views

How to train Ner Model having 1 entity?

I am Creating a Custom NER (named entity recognition ) Model using bi directional LSTM and CRF. During Study on Ner i see all example includes Multiple entities per sentence. For eample this ...
0
votes
0answers
49 views

How to deal with missing values in LSTM? Masking

I try to use LSTM for time series data prediction. The input data has the following form. missing valueBut I have some problem to use the Masking to deal with the data. Because for (samples,timestep,...
2
votes
0answers
58 views

Setting up RNN in TensorFlow for time series forecast with variable input series lengths

I am building a model with keras for time series prediction. The structure of the problem is as follows: The input is a time series of 5 numeric features The ...
1
vote
1answer
32 views

About seq2seq networks

I have read that seq2seq is a network, similar to other networks types (CNN, RNN, ...). However, in my opinion it is actually an architecture for RNNs. Isn't that? For example, when input and output ...
1
vote
1answer
59 views

impact of varying sequence length in ensemble GRU model

I am using ensemble gru for my project and keeping different cell sizes for different models !For example, first gru model is of size 16 and the second is of 8 and 4 for the third model. The model is ...
2
votes
0answers
107 views

Using LSTM to forecast vehicle position - multivariate time series - Matlab

I am trying to train an LSTM model on Matlab to forecast the position of a vehicle when driving around a roundabout. My main concern right now is that my dataset consists of 4 features (X position, Y ...
2
votes
1answer
283 views

building a 2-layer LSTM for time series prediction using tensorflow

From Tensorflow tutorials i am experimenting time series with LSTM In the section 'multi-step prediction' using LSTM tutorial says Since the task here is a bit more complicated than the previous ...
0
votes
1answer
365 views

TypeError: object of type 'Bidirectional' has no len()

I'm running a bidirectional LSTM. But this error is appearing: TypeError: object of type 'Bidirectional' has no len() What's wrong in this code? Please help. ...
2
votes
0answers
22 views

Wiggle in the initial part of an LSTM prediction

I working on using LSTMs and GRUs to make time series predictions. For the most part the predictions are pretty good. However, there seems to be a wiggle (or initial up-then-down) before the ...
0
votes
1answer
659 views

How to calculate perplexity in PyTorch?

I am wondering the calculation of perplexity of a language model which is based on ...
0
votes
1answer
69 views

should I shift a dataset to use it for Time series regression with RNN/LSTM?

I'm seeing this tutorial to know how to use LSTM to predict time series data and I noticed that he shifted the target/labels up so that the features are all in time t but the target is t+1 so my ...
0
votes
0answers
17 views

Train MountainCar-v0 with LSTM

In mountain car envoriment we can accelerate car to left or right. When I get random samples for training, I'm getting 2:1 ratio for acceleration to left. When I train my LSTM model, it reaches ~ 70% ...
2
votes
2answers
86 views

Recommended data cleaning techniques for multivariate time series prediction?

I have to predict the next step(s) in a multivariate time series with about 30 features and 50.000 samples. I am thinking of using LSTM. Which techniques are usually recommended for cleaning the data ...
1
vote
0answers
81 views

Issue with implementation of Sequence to Sequence Autoencoder

Hi, I want to implement Sequence to Sequence Autoencoder for text message classification purpose whether it is spam or ham. The complete code is: ...
1
vote
1answer
56 views

How to get vectors of memory cell & the last output of $LSTM$ in keras?

In this research paper the following paragraph appears, The state of every LSTM model is stored in two fixed-size vectors of real numbers called the memory cells and the last output. Since our ...
-1
votes
1answer
52 views

Can a machine learning model be trained on Call Detail Record(CDR) Data to predict user's daily locations?

I have a CDR data for two months and my goal is to extract daily or frequent locations(cell towers) of the user along with the departure and arrival time on those locations. The spatial resolution of ...
0
votes
1answer
373 views

LSTM with linear activation function

I'm trying to do multi-step regression and I use an output layer: LSTM(1, activation='linear', return_sequences=True) Is this the wrong way of achieving this? Should I use a TimeDistributed(Dense(1))...
1
vote
0answers
38 views

Using Keras/TS for Multivariate Time Series Prediction w/ Univariate output?

I've been reading through a few tutorials for using Keras/TensorFlow for multivariate time-series prediction (primarily using LSTM models). One example uses air pollution as an example. In this ...
2
votes
2answers
208 views

Training LSTM for time series prediction with nan labels

I have a time series of features $x_1,x_2,x_3,...,x_n$. I want to make a prediction $y_1,y_2,y_3,...,y_n$ for each timestep. However, in my training data some of the $y$ can be nan. I'd like the fit ...
2
votes
0answers
56 views

Training an LSTM with different time steps and number of features

I want to use an LSTM using Keras to make course grade predictions. My dataset includes student transcripts, which consist of courses taken and their respective grades of students. For each course, I ...
0
votes
0answers
31 views

LSTM on Multiple Time Series fails to complete training

I am working on a sales forecasting model using LSTM from Keras with Tensorflow backend. My data consists of 204 products that are picked based on an Adjusted R Squared of >=0.5. I have trained the ...
0
votes
0answers
8 views

Output of a seq2seq network

I've created a seq2seq lstm that uses word embeddings, following the tutorial in tutorial , but I'm having some problems understanding the output, which is along these lines: ...
1
vote
1answer
166 views

Keras LSTM Input Shape - Batch Size and Time Step

So I have 82 different sets of data, each with varying length where each point has one feature and a label (0 or 1). I'm trying to use Keras LSTM to be able to predict the class of a point depending ...
3
votes
2answers
76 views

why an advanced LSTM model produce the same results as a simpler one?

I have implemented the model proposed in this article which is a text classification model that uses sentence representation rather than only word representation to classify texts. ...
0
votes
2answers
47 views

How to apply a different Loss function to one specific Label?

I got a recurrent neural network in Keras, which classifies on 14 labels. The first label is the most important one and should be predicted with the highest accuracy. The other labels don't have to be ...
1
vote
1answer
53 views

Customize loss function for Music Generation LSTM (?)

I have to carry out a Music Generation project for a Deep Learning course I have this semester and I am using Pytorch. The dataset is songs in midi format and I use the python library mido to extract ...
0
votes
0answers
145 views

How to calculate the memory usage of a deep LSTM network?

I was trying to estimate the memory usage for my LSTM network by referring to an examples of CNN memory usage calculation at http://cs231n.github.io/convolutional-networks/#computational-...
0
votes
0answers
97 views

how or why or when to combine Dense() and LSTM()-layers?

I often see topologies like the following: and I wonder why are there Dense()-layers at all? When or how or why do I make a decision to put a Dense()-layer in between LSTM()-layers? Or maybe also the ...
0
votes
0answers
58 views

using reinforcement learning for classification

this is a purely conceptual query and in case the moderators feel it needs to be asked elsewhere, i would be happy to move it there. We do a lot of work in text classification and a senior ...
0
votes
1answer
58 views

Apply LSTM to each matrix element with Keras

I'm trying to apply a LSTM/GRU to each entry of a matrix $X$ note: Each matrix element is a time-series, so shape of X is (batch_size, rows, cols, time_steps, dims) $ y_{i,j}= \begin{cases} ...
0
votes
0answers
17 views

Many to One LSTM; scientific Term

What is the Scientific Term for a Many to One LSTM ? Picture from here
2
votes
1answer
33 views

Multiple merging multiple convolutions

(First post here) I am rather new to neural networks, having used Tensorflow for a couple months now, and am looking for some advice I have on an idea to improve the accuracy of my model. I am looking ...
1
vote
0answers
28 views

Improving Performance of LSTM for time series prediction

My data consists of two features and a set of time series data labeled as "bookings". I have 1056 data point in the times series, for which I have two features for each. The data size is 1056x3. My ...
0
votes
0answers
107 views

How to add new csv file data into training LSTM model to predict next future value using python

Here I have a data csv file with four inputs. I want to predict next value using LSTM model. first of all I train the LSTM model with data. Here is my code: ...
0
votes
0answers
256 views

Weather Forecasting: CNN-LSTM or ConvLSTM?

I am trying to develop a weather forecast model where satellite images (temperature, velocity field etc) are stacked over time. Since the prediction model needs to analyze both spatial features and ...
0
votes
1answer
60 views

Understanding LSTM Keras implementation

So I understand what LSTM units are. But I have trouble understanding the implementation / function in Keras framework. Let's say, I add a layer ...
0
votes
0answers
178 views

Adapting Pytorch tutorial “NMT from Scratch…” for dynamic RNN

I have taken the code from the tutorial and attempted to modify it to include bi-directionality and any arbitrary numbers of layers for GRU. Link to the tutorial which uses uni-directional, single ...
0
votes
0answers
18 views

Bi-Directional LSTM/GRU better than LSTM/GRU?

Is there any research paper or blog post that discusses 'is Bi-Directional LSTM/GRU better than LSTM/GRU' under scenario and data set?
0
votes
0answers
17 views

Effective Time Series Forecasting using Keras/LSTM

I am working on time series forecasting for an engineering component (turbo charger). I have dataset containing field data from sensors (=features) taken every day for different turbocharger for their ...
1
vote
0answers
53 views

chatbot encoder/decoder: why do we need to use chatbot answer as the decoder inputs?

I am looking into the chatbot tutorial at: https://medium.com/predict/creating-a-chatbot-from-scratch-using-keras-and-tensorflow-59e8fc76be79 It uses sequence to sequence model with encoder/decoder ...
1
vote
1answer
36 views

Time Series Classification for 1 hour blocks

I am doing some analysis on time series. The time series would consist of 3 channels and contain 5 minute interval data. What I want is to be able to give it a 1 hour block of 5 minute interval data ...
3
votes
4answers
1k views

What is difference between feed forward neural network and LSTM?

What is the difference between feed-forward neural network and LSTM? How do they differ in their architecture?

1
3 4
5
6 7
17