Questions tagged [lstm]
LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.
709
questions
0
votes
0answers
3 views
structuring time dependent features classification
I have a binary classification task at hand to predict Status = 1,0. The independent variables are a function of time, with each row representing a task that updates the status if things change . in ...
0
votes
0answers
11 views
LSTM autoencoder reconstructs input in ascending order
I implemented an autoencoder LSTM using Keras just as indicated in this article: article. The problem is that the reconstructed input of the time-series is given in ascending order with respect to the ...
1
vote
1answer
30 views
How to get feature importance from a keras deep learning model?
In case of scikit-learn's models, we can get feature importance using the relevant attributes of the model.
I've been working on a RNN, using LSTMs for text embedding.
Is there any way to get ...
1
vote
0answers
8 views
How can I do a sequence to sequence model (RNN / LSTM) with Keras with fixed length data?
What I'm trying to do seems so simple, but I can't find any examples online. First, I'm not working in language, so all of the embedding stuff adds needless ...
0
votes
0answers
9 views
Same Output for every row when doing prediction. Thus I am having wrong predictions
I have implemented an emotion analysis using lstm method, I have trained my model in the first part of the codes and i am now am doing the prediction part. When I run my system, I am having same ...
1
vote
0answers
13 views
How to return states of LSTM in MXNet?
I used MXNet previously to beat keras+tensorflow accuracy in CNN regression models.
Now I am trying to implement LSTM, which in keras runs fine:
...
2
votes
1answer
23 views
Sentences language translation with sequential layers neural network
Context: Many language sentences translation systems (e.g. French to English) with neural networks use a seq2seq structure:
"the cat sat on the mat" -> [Seq2Seq ...
0
votes
1answer
25 views
Understanding the concept vanishing gradient and exploding gradient problem in terms of training data
I'm trying to figure out the essence of the concepts "vanishing gradient and exploding gradient problem" in terms of real-world input-output training examples instead of in terms of the properties of ...
1
vote
0answers
19 views
How to setup LSTM problem when using multiple time series inputs?
Problem:
Suppose I have 50 time series which I want to train on. For each series, given the last say 4 samples, I would like to predict the next value.
Suppose each series is 100 samples. I break ...
1
vote
0answers
16 views
Time Series Forecasting with RNN/LSTM/NARX
I have some experimental datasets (like 4 or 5), and each dataset has three time series data, say $u1(t)$, $u2(t)$, and $x(t)$. The three time series of each experiment are similar but not the same. ...
2
votes
1answer
16 views
Use LSTM to predict the proportion of steps with nonzero feature values
I am trying to do a simple regression for sequences. Each input $X_i$ is a $n=2000$ by 1 matrix, formatted as $n_i$ 0-s followed by $(n-n_i)$ 1-s. The output $y_i$ should be $n_i/n$, i.e. the ...
0
votes
0answers
18 views
What does linearly decreasing training and accuracy loss means?
I am trying to train an LSTM autoencoder for sentence embeddings:-
...
0
votes
1answer
27 views
What is the output of multivariate LSTM model?
I am currently trying to build an LSTM model by using multivariate inputs, but I don't understand what exact output I am predicting.
I am currently using 5 features in the data i.e. 'Time', 'Avg CPU ...
1
vote
1answer
24 views
How can I tune LSTM hyperparameters?
If anyone is there to answer these, that'll be great. I'm in the midst of a Final Year Project on LSTM.
Currently, I’m stuck and confused over LSTM codes. There are 4 hyperparameters that I can play ...
0
votes
0answers
37 views
Root cause analysis using Machine Learning or NN
I am trying to build a predictive model that will analyze pattern from a snapshot file (log) and give the root cause analysis of the file i.e the class to which the pattern in the logs belongs to.
I ...
2
votes
0answers
22 views
LONG SHORT-TERM MEMORY Hochreiter's paper BPTT
I was trying to read paper about LSTM, and I am stuck with mathematical problem.
http://www.bioinf.jku.at/publications/older/2604.pdf
page 4.
see,
|$f'_l$$_m$($net_l$$_m$)$w_l$$_m$ $_l$$_m$$_-$$_1$...
1
vote
1answer
79 views
Using GRU with FeedForward layers in Python
I'm trying to reproduce the codes in this paper here for the multi-labeling problem (11 classes), which is using
...
1
vote
0answers
29 views
Loss is decreasing correctly but upon prediction, totally wrong results
I've made a pretty basic stock prediction RNN with it's only input being the past stock price from apple. On this case, I want to input the apple stock price and the samsung stock price (2 features). ...
1
vote
1answer
48 views
Predicting parallel time series with multiple features
I am trying to predict sales for 2 departmental stores which share similar demographic properties. My goal is to make a single LSTM model to predict sales from these parallel time series having ...
2
votes
1answer
30 views
Can I use LSTM models to evaluate multiple, independent time series?
Let's say that I would like to predict the temperature tomorrow. I could use the approach whereby I train a model based on a time-series dataset collected from a single location (for example, see this ...
2
votes
1answer
35 views
Word Embedding or Hash?
In my dataset I have a 'text' column and a 'followers' column containing lists of follower IDs, i.e. '1093777852477116417, 936194589043683328,...'. Some of the 'followers' values contain thousands of ...
-1
votes
1answer
30 views
validation accuracy and loss increase
I am training a generic LSTM based autoencoder to get the sentence embeddings, the bleu score is the accuracy metric. The model is coded to output the same number of tokens as the length of labels, ...
1
vote
0answers
23 views
Help understanding the Tensorboard histogram names and meaning in an LSTM Model
Can someone please help me understand what the names and shapes of the following tensorboard histogram outputs mean about an LSTM model I coded? Thank you!
I understand the terms in the names like ...
1
vote
0answers
35 views
TensorFlow 2: Find MAE, RMSE for validation dataset in time-series LSTM
TensorFlow 2 "Time series forecasting tutorial" (https://www.tensorflow.org/tutorials/structured_data/time_series#recurrent_neural_network) gives an example of a LSTM multi-step prediction model that ...
0
votes
1answer
29 views
Keras RNN (batch_size
I created RNN model for text classification with LSTM layer, but when I put the batch_size in the fit method, my model trained on the whole batch instead of just the mini batch _size.
This also ...
1
vote
2answers
30 views
h in LSTM increasing in size?
So I was reading about the LSTM architecture and I was having trouble understanding a certain aspect of it. This article mentions the step in question near the bottom of the page. Here is the image ...
2
votes
2answers
89 views
What is the role of $W_{ax}, W_{aa}, W_{ay}$ in forward propagation in RNN? Are they hyperparameters? Why are they needed?
In RNN introduction in Coursera sequence model course, the following formula for forward propagation in RNN was introduced. What exactly is the role of $W_{ax}, W_{aa}, W_{ay}$? What do they do?
In ...
1
vote
1answer
31 views
Using LSTMs for continous learning and predicting
I'm trying to develop a model to predict a commodity price movement direction based on previous observations. The model should learn common technical analysis patterns, e.g. head and shoulders. So, I ...
6
votes
1answer
63 views
Activation function between LSTM layers
I'm aware the LSTM cell uses both sigmoid and tanh activation functions internally, however when creating a stacked LSTM architecture does it make sense to pass their outputs through an activation ...
2
votes
0answers
18 views
Dense? or TimeDistributedDense? after LSTM layer in Keras
Dense and TimeDistributedDense, which one is suitable after LSTM layer in Keras?
For example,
...
2
votes
0answers
21 views
Masking seems not working for missing values problem in LSTM
I am trying to use LSTM to predict time series in keras. My input data shape is (1000,6,1)(samples,timesteps,features). There is some missing data in different timesteps. For example,[2,1,1]=NaN,[3,4,...
3
votes
1answer
66 views
TensorFlow / Keras: What is stateful = True in LSTM layers?
Could you elaborate on this argument? I found the brief explanation from the docs unsatisfying:
stateful: Boolean (default False). If True, the last state for each sample at index i in a batch will ...
2
votes
1answer
31 views
How to train Ner Model having 1 entity?
I am Creating a Custom NER (named entity recognition ) Model using bi directional LSTM and CRF.
During Study on Ner i see all example includes Multiple entities per sentence.
For eample this ...
0
votes
0answers
21 views
How to deal with missing values in LSTM? Masking
I try to use LSTM for time series data prediction. The input data has the following form.
missing valueBut I have some problem to use the Masking to deal with the data. Because for (samples,timestep,...
2
votes
0answers
46 views
Setting up RNN in TensorFlow for time series forecast with variable input series lengths
I am building a model with keras for time series prediction. The structure of the problem is as follows:
The input is a time series of 5 numeric features
The ...
1
vote
1answer
30 views
About seq2seq networks
I have read that seq2seq is a network, similar to other networks types (CNN, RNN, ...). However, in my opinion it is actually an architecture for RNNs. Isn't that?
For example, when input and output ...
1
vote
1answer
32 views
impact of varying sequence length in ensemble GRU model
I am using ensemble gru for my project and keeping different cell sizes for different models !For example, first gru model is of size 16 and the second is of 8 and 4 for the third model.
The model is ...
2
votes
0answers
43 views
Using LSTM to forecast vehicle position - multivariate time series - Matlab
I am trying to train an LSTM model on Matlab to forecast the position of a vehicle when driving around a roundabout. My main concern right now is that my dataset consists of 4 features (X position, Y ...
2
votes
1answer
66 views
building a 2-layer LSTM for time series prediction using tensorflow
From Tensorflow tutorials i am experimenting time series with LSTM
In the section 'multi-step prediction' using LSTM tutorial says
Since the task here is a bit more complicated than the previous ...
0
votes
1answer
38 views
TypeError: object of type 'Bidirectional' has no len()
I'm running a bidirectional LSTM. But this error is appearing:
TypeError: object of type 'Bidirectional' has no len()
What's wrong in this code? Please help.
...
2
votes
0answers
17 views
Wiggle in the initial part of an LSTM prediction
I working on using LSTMs and GRUs to make time series predictions. For the most part the predictions are pretty good.
However, there seems to be a wiggle (or initial up-then-down) before the ...
0
votes
1answer
114 views
How to calculate perplexity in PyTorch?
I am wondering the calculation of perplexity of a language model which is based on ...
0
votes
1answer
24 views
should I shift a dataset to use it for Time series regression with RNN/LSTM?
I'm seeing this tutorial to know how to use LSTM to predict time series data and I noticed that he shifted the target/labels up so that the features are all in time t but the target is t+1
so my ...
0
votes
0answers
8 views
Train MountainCar-v0 with LSTM
In mountain car envoriment we can accelerate car to left or right.
When I get random samples for training, I'm getting 2:1 ratio for acceleration to left.
When I train my LSTM model, it reaches ~ 70% ...
2
votes
2answers
58 views
Recommended data cleaning techniques for multivariate time series prediction?
I have to predict the next step(s) in a multivariate time series with about 30 features and 50.000 samples. I am thinking of using LSTM. Which techniques are usually recommended for cleaning the data ...
1
vote
0answers
35 views
Issue with implementation of Sequence to Sequence Autoencoder
Hi,
I want to implement Sequence to Sequence Autoencoder for text message classification purpose whether it is spam or ham.
The complete code is:
...
1
vote
1answer
32 views
How to get vectors of memory cell & the last output of $LSTM$ in keras?
In this research paper the following paragraph appears,
The state of every LSTM model is stored in two fixed-size
vectors of real numbers called the memory cells and the last
output. Since our ...
-1
votes
1answer
25 views
Can a machine learning model be trained on Call Detail Record(CDR) Data to predict user's daily locations?
I have a CDR data for two months and my goal is to extract daily or frequent locations(cell towers) of the user along with the departure and arrival time on those locations. The spatial resolution of ...
0
votes
1answer
34 views
LSTM with linear activation function
I'm trying to do multi-step regression and I use an output layer:
LSTM(1, activation='linear', return_sequences=True)
Is this the wrong way of achieving this?
Should I use a TimeDistributed(Dense(1))...
1
vote
0answers
22 views
Using Keras/TS for Multivariate Time Series Prediction w/ Univariate output?
I've been reading through a few tutorials for using Keras/TensorFlow for multivariate time-series prediction (primarily using LSTM models).
One example uses air pollution as an example. In this ...
