Questions tagged [lstm]
LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.
428 questions
2
votes
0answers
7 views
Can somebody explain me the the following parameters of Keras LSTM layer
keras.layers.LSTM(units,stateful=False,unroll=False)
What units,stateful and unroll represents here??
0
votes
1answer
18 views
Model for classifying time-series data with distinct features?
I've heard about time-series classification being done with TCN's and CNN's combined with LSTM's very often, citing that CNN's would provide insight both forward and in the past since you already have ...
1
vote
0answers
23 views
Zero padding for LSTM input
I am building a text-generation model. In the first layer, I am using Word2Vec embeddings. Now since the input is sentences they are variable length and I am padding them with zero. The input is ...
0
votes
0answers
7 views
Split timeline for training LSTM network
I have faced some trouble in splitting my dataset before feeding the data into an LSTM network. My data are time series, so that the only feasible way is to split them into training and testing part ...
1
vote
1answer
18 views
Books on time series and sequence classification
Though I have been using traditional machine learning algorithms (Regression and Classification) , I have no experience of using Time series and would like to understand what is time series and ...
2
votes
2answers
126 views
Unnormalized Log Probability - RNN
I am going through the deep learning book by Goodfellow. In the RNN section I am stuck with the following:
RNN is defined like following:
And the equations are :
Now the $O^{(t)}$ above is ...
1
vote
0answers
12 views
Tensor Shape 'Reshape' during the dataflow
I would like to combine an Autoencoder with a LSTM. However, the 'timestep' is a block for the implements and I would like to train them together. Is there a solution to the tensor 'reshape'? I mean, ...
1
vote
1answer
20 views
Why increasing the number of units or layers does not increase the accuracy and decrease the loss?
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
...
3
votes
0answers
49 views
+50
convLSTM : how to structure input data
I have the following dataframe containing training data that I have been using to perform a regression task using CNN + FC :
...
0
votes
0answers
15 views
Remedies to CNN-LSTM overfitting on relatively small image dataset
Notes
Using a pretrained model, trying data augmentation (not possible knowing nature of images, lowering number of parameters in the network, all didn't help)
Context
I have a sequence of images. ...
1
vote
1answer
30 views
What will happen if the number of units and the time-steps are different numbers in LSTM?
How will data feed the LSTM in following scenarios?
I have a data array with the shape of (100,10,3) and a ...
0
votes
0answers
17 views
Loss function minimizing by pushing precision and recall to 0
I am designing a Recurrent Neural Network with LSTM cells (3, 5, 3 at the moment) to classify highly skewed data using the keras framework with Python. There are 8,640 time steps per day (10 second ...
0
votes
0answers
13 views
Can I have multiple (a sequence of) predicted values as the output of a neural network model?
I have a multi-dimensional time series data, and I want to use these data to do a time-series prediction. That is, the target(ground truth) of the training data is in a time series format instead of ...
2
votes
2answers
51 views
understanding linear algebra of a forget gate
This blog covers the basics of LSTMs.
A forget gate is defined as :
$$f_t = \sigma(W_f \cdot [h_{t-1}, x_t]+ b_f)$$
At this point the linear algebra confuses me more than it should. The syntax of $...
2
votes
1answer
30 views
Is normalizing the validation set of time series a kind of look ahead bias?
Here's the data normalization process of a time series in a paper about stock prediction using LSTM:
Split train and test set based on time (e.g. training set: 2001-2010, test set:2011-2012). This ...
0
votes
1answer
21 views
Loss is decreasing but val_loss not! [duplicate]
If loss is decreasing but val_loss not, what is the problem and how can I fix it?
I get such vague result:
1
vote
1answer
73 views
LSTM forecasting on multivariate time series
I'm new to RNNs and LSTM and would like some direction with a problem I have. I have a data set containing system metrics (like CPU utilization, disk operations, memory use) of an AWS EC2 instance ...
0
votes
0answers
21 views
Filtering data before passing through LSTM then returning the filtered data back into the results
I have a training dataset with two classes (0,1 but future datasets will have 3 classes) and a testing dataset with three classes (-1, 0, 1). My problem is one of binary classification however, the ...
0
votes
0answers
15 views
Is it possible to have too few labels to train a model?
I’m trying to train seq2seq LSTM model to recognize “good” and “bad” bits in samples of raw, real timeseries, using synthetic training data. While “good” data has some sort of consistency (based on ...
0
votes
1answer
54 views
The memorisation capacity of an LSTM (real numbers)
My question is the following:
It is known that a LSTM can remember sequences of one-hot encodings which represent integers (i.e. output $x_1, ... x_n$ after receiving $x_1, ... x_n$ as inputs, $x_k \...
0
votes
0answers
19 views
How to training the recurrent recommender system with LSTM?
Recently, I read a paper about recurrent recommender system, I am very curious about how it training its network.
Assume I have the Netflix dataset as
...
0
votes
1answer
30 views
How to design a many-to-many LSTM?
I have an input array of shape (1000,20, 4) and output(labels) of shape (1000,25,1). But don't know how to use Keras LSTM library to build a sequential model for this!
Can someone help me design a ...
1
vote
0answers
21 views
Predicting Wave Trends of Candelestick Charts in Tensorflow (JS)
I'm relatively new to ML but my goal it to use Tensorflow.js and build a ML model that can help me detect a certain wave formation for an automated trading system. Examples of the 3-leg pattern I am ...
1
vote
1answer
13 views
What does it mean: “Everything looks OK but loss won't decreases!”
I have written a LSTM network. It seems all the things are OK but when I train the network, I get the same loss amount about 4.9e-4 for every iterations! What is the problem? Why my network can't ...
1
vote
1answer
16 views
Why do recurrent layers work better than simple feed-forward networks?
On a time series problem that we try to solve using RNNs, the input usually has the shape $input features \times timesteps \times batchsize$ and we then feed this ...
0
votes
1answer
26 views
Is the LSTM share the same model parameter in each block?
I learn the LSTM recently, and a little bit confuse about the model parameters about LSTM,
The follow is the LSTM structure
And it is equation as (I slightly ignore the bias in each equation):
$$...
0
votes
0answers
11 views
Small output range and delayed output? Predicting sine using LSTM
I have coded a very basic LSTM with forget gates (no libraries used). I'm trying to predict $0.5\cdot \sin(t + N)$ given $0.5\cdot\sin(t)$ as an exercise.
I have tweaked the model, changing the ...
0
votes
0answers
6 views
Does LSTM just mocking previous data?
I am trying to do a numerical time series prediction using LSTM but it seems that the LSTM just mocking previous data(Even when ...
2
votes
1answer
17 views
What are the output shape of LSTM with “return_sequences” equal to “True” or “False”?
What are the output shape of LSTM with Keras implementation when "return_sequences" equals to "True" or "False" ?
0
votes
1answer
23 views
What is the job of “RepeatVector” and “TimeDistributed”?
I read about them in Keras documentation and other websites, but I couldn't exactly understand what do exactly they do and how should we use them in designing ...
0
votes
1answer
40 views
ML technique to predict next performance anomaly
I would like a direction for a ML technique to predict when an anomaly will occur in a system, like when the CPU use differs from a normal state.
My data consists of system metrics collected from AWS ...
1
vote
0answers
14 views
How does it work if we have different “time-steps” and “LSTM-units”?
As I know we can create a LSTM layer like:
model.add(LSTM(units), input_shape = (time-steps, feature_number))
And the ...
2
votes
1answer
24 views
How can I specify I want output of which units in LSTM?
I see this code concept(with Keras library) in most code examples of LSTM:
model.add(LSTM(X))
model.add(Dense(Y))
But I don't ...
0
votes
1answer
15 views
Difference between “reducing batch_size” and “increasing epochs” to decrease loss amount?
In my experience, both reducing batch_size and increasing epochs can decrease loss amount. But I like to know is there any ...
0
votes
0answers
35 views
Autoencoder for Dimensionality Reduction
My task is to reduce the features of my temporal sequence. Each input is of the shape (timesteps, features) = (240,117). I am using an autoencoder consisting of intermediate lstm layers and I am ...
2
votes
2answers
44 views
What is the difference between “Adding more LSTM layers” or “Adding more units on existence layers”?
What is the difference between adding more LSTM layers and just increasing the units of existing layers? Which one is preferred and in which situation?
0
votes
1answer
22 views
Keras: extreme spike in loss during training
I am training an LSTM for time series forecasting and it has produced an extremly high loss value during one epoch:
...
2
votes
1answer
16 views
What is the difference? “Adding more LSTM layers” or “Increasing epochs”?
To get more accurate results which one is better? Adding more layers or increasing number of epochs? I like to know the difference between effects of these two approaches?
2
votes
1answer
24 views
Papers on anger detection in dialogues
I am interested in anger detection in dialogues and I want to study multiple methods like LSTM, CNN, etc. Are there any good research papers or books about this subject?
1
vote
1answer
40 views
Is there any standard or normal range for the amount of LSTM loss function?
I am working on a LSTM network that I get loss amounts around 4.7 e-4 . It seems adding more layers and increasing epochs don't help to decreasing it. I also using a ...
2
votes
0answers
18 views
NN architecture for highly repetitive, discrete, but noisy data?
I have some multivariate, noisy timeseries where some combination of the features results in a square-form like signal. However, the signal contains a lot of noise, unwanted wiggles and sometimes the ...
0
votes
1answer
70 views
What is the vector value of [CLS] [SEP] tokens in BERT
In BERT, They replace separator and start of sentence with special token labels. What are there corresponding values in embedding_matrix. Are they 0-vector?
I wanted to replace the proper nouns like ...
0
votes
1answer
28 views
Is “adding the predictions to the real data for new training and prediction” a good idea for LSTM?
Considering we have trained our model with a lot of data for "many-to-one" prediction. Then we like to forecast the future data of next 10 days. So we use last 60 of existent data and predict the ...
0
votes
0answers
12 views
Class weights for time-series data with imbalanced classes
I am having an issue where my loss is decreasing on every epoch, but my precision and recall are still extremely small. This is, in my understanding, because I have time-series of length 8639 and each ...
1
vote
2answers
37 views
extraction information from resume
I have a project in machine learning in which I need to analyze a
curriculum vitae. for that I have to write a python program. It uses
basic techniques of Natural Language Processing like word
...
1
vote
1answer
39 views
Should I only use scalar labels with Keras LSTM?
I have an array X_train = (1110,25,2) and a y_train = (1110,5,2). It means I use arrays with length of 25 for inputs and length ...
4
votes
1answer
24 views
Why my results have time delay when I use LSTM?
I am trying to fit and test LSTM on a numeric series(like stock prices). But it seems that I always get a lag in predicted graph(Blue) with respect to real graph(red). Does anyone know why this ...
0
votes
0answers
15 views
Do 3D-CNN suffer from exploding and vanishing gradient?
I am building a video classification network, I see 2 options for the same CNN+LSTM and 3D-CNNs, Do 3D-CNNs suffers from exploding and vanishing gradients like LSTMs or are 3d-CNN better at handling ...
1
vote
1answer
17 views
Which scaling size is better? [0,1] or [-1,1] for LSTM?
I see some scale their data between 0 and 1 and some others do that between -1 and 1. But which one is better? Or better to ask: How to make a true/good decision for that?
1
vote
1answer
25 views
What is the difference between different batch_sizes in Keras Sequential models?
I am interested to know, what happens when I choose batch_size=1 or batch_size=1000 or any other numbers in ...
