Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

Filter by
Sorted by
Tagged with
5
votes
2answers
3k views

Understanding input of LSTM

I am a little confused with the input of LSTM. Basicaly my train input data is of shape (53394, 3). I reshaped my 2D data into 3D data in order to set it according to the input of LSTM. I have two ...
5
votes
2answers
4k views

Initial embeddings for unknown, padding?

Last time I've been passing pretrained word embeddings into LSTM to solve text classification problems. Usually, there are additional <pad>, ...
5
votes
1answer
4k views

How to implement LSTM with Spark?

I would like to build an LSTM network for text classification with PySpark, but I don't find any library or function about it. ...
5
votes
2answers
1k views

Anomaly detection using RNN LSTM

I'm trying to detect anomalies in an univariate time series. I trained a RNN LSTM and currently I get one-step-ahead predictions. Could someone explain if it's possible to output a confidence ...
5
votes
1answer
196 views

Suspected Exploding Gradient in Character Generator LSTM

I'm trying to create a neural network that can learn how to write text character by character from the book David Copperfield (via Project Gutenburg). It starts great, then forgets punctuation ...
5
votes
1answer
50 views

N-grams for RNNs

Given a word $w_{n}$ a statistical model such a Markov chain using n-grams predicts the subsequent word $w_{n+1}$. The prediction is by no means random. How is this translated into a neural model? I ...
5
votes
1answer
175 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
4
votes
2answers
4k views

Relationship between batch size and the number of neurons in the input layer

Regarding LSTM neural networks, I am unable to understand the relationship between batch size, the number of neurons in the input layer and the number of "variables" or "columns" in the input. (...
4
votes
3answers
857 views

Why using a frozen embedding layer in an LSTM model

I'm studying this LSTM mode: https://www.kaggle.com/paoloripamonti/twitter-sentiment-analysis They use a frozen embedding layer which uses an predefined matrix with for each word a 300 dim vector ...
4
votes
1answer
99 views

Which target variable should I use?

I have a problem where I want an LSTM to predict the resistance of a body. This value can also be calculated if we know the drag coefficient and the speed of that body. In my case, at inference time, ...
4
votes
2answers
884 views

Understanding LSTM input shape for keras

I am learning about the LSTM network. The input needs to be 3D. So I have a CSV file which has 9999 data with one feature only. So it is only one file. So usually it is ...
4
votes
1answer
2k views

Why is predicted rainfall by LSTM coming negative for some data points?

I have used supervised learning with LSTM network using tanh activation function and 0.1 dropout for time series prediction.my loss='mean_squared_error', optimizer='adam'. The predicted time series is ...
4
votes
3answers
1k views

Handwritting Recognition moving from character level to word level

Given the experience on MIST, I try this problem as a character level. I have a handwritten text and I want to "OCR" it. Even though I made progresses with openCV (on the image pre-processing, ...
4
votes
1answer
2k views

Forecasting via LSTM or XGBoost… is it really a forecast or

I guess I understand the idea of predictions made via LSTM or XGBoost models, but want to reach out to the community to confirm my thoughts. This tutorial does a nice job explaining step by step of ...
4
votes
1answer
73 views

Do timesteps must have the same temporal distance in training a RNN?

I have a recurrent neural network with LSTM units that I want to train with batches of 6 timesteps. Each timestep is a record of a dataset and represents the temporal aggregation over 5 minutes of ...
4
votes
1answer
76 views

Why my results have time delay when I use LSTM?

I am trying to fit and test LSTM on a numeric series(like stock prices). But it seems that I always get a lag in predicted graph(Blue) with respect to real graph(red). Does anyone know why this ...
4
votes
1answer
4k views

Is there an R tutorial of using LSTM for multivariate time series forecasting?

There is a great blog post about how to use keras stateful LSTM in R to forecast sunspots. I applied it to financial ts data ...
4
votes
2answers
334 views

RNN in pseudo-code

A few years ago, I understood the classical MLP neural network much better when I wrote an implementation from scratch (using only Python + Numpy, without using tensorflow). Now I'd like to do the ...
4
votes
2answers
538 views

Can I use LSTM models to evaluate multiple, independent time series?

Let's say that I would like to predict the temperature tomorrow. I could use the approach whereby I train a model based on a time-series dataset collected from a single location (for example, see this ...
4
votes
2answers
822 views

Trying to understand encoder-decoder sequential models in Keras?

My understanding is that for some types of seq2seq models, you train an encoder and a decoder, and then you set aside the encoder and use only the decoder for the prediction step. For example this ...
4
votes
1answer
110 views

How to predict value in every 120 minutes using LSTM in python

I want to predict value in every 120 minutes continuous using LSTM model. Here I wrote the code for predction. But I'm not getting proper prediction values . Here from start time I need to predict ...
4
votes
1answer
2k views

In Keras, how to get 3D input and 3D output for LSTM layers

In my original setting, I got X1 = (1200,40,1) y1 = (1200,10) Then, I work perfectly with my codes: ...
4
votes
1answer
524 views

Do I need to engineer lagged features when creating an LSTM for time series forecasting?

Long short-term memory networks are fairly complicated and I haven't completely wrapped my head around them. It seems to me like the big gain in LSTMs for time series forecasting is the lacking ...
4
votes
1answer
219 views

On the choice of LSTM input/output dimension for a spatio-temporal problem

I am using LSTM neural networks from (R)Keras for a matter of spatio-temporal interpolation. I manage to get the network to output predictions but the results are not outstanding (very little ...
4
votes
1answer
103 views

Is it a red flag that increasing the number of parameters makes the model less able to overfit small amounts of data?

I'm training a deep network (CNN-LSTM-CRF) for Named Entity Recognition. Is there a reason that increasing the number of parameters would make the network less able to overfit a small training set (~...
4
votes
1answer
2k views

Autoencoders for the compression of time series

I am trying to use autoencoder (simple, convolutional, LSTM) to compress time series. Here are the models I tried. Simple autoencoder: ...
4
votes
1answer
676 views

Train LSTM model with multiple time series

I am predicting energy usage for a bedroom within a school residential building with date, temperature, and humidity as input features, using 7 time-steps and ...
4
votes
4answers
1k views

Understanding output of LSTM for regression

I am working with embeddings and wanted to see how feasible it is to predict some scores attached to some sequences of words. The details of the scores are not important. ...
4
votes
0answers
111 views

LSTM Long Term Dependencies Keras

I am familiar with the LSTM unit (memory cell, forget gate, output gate etc) however I am struggling to see how this links to the LSTM implementation in Keras. In Keras the input data structure for X ...
4
votes
0answers
1k views

Using the Python Keras multi_gpu_model with LSTM / GRU to predict Timeseries data

I'm having an issue with python keras LSTM / GRU layers with multi_gpu_model for machine learning. When I use a single GPU, the predictions work correctly ...
4
votes
0answers
3k views

Kalman filter for time series prediction

I have the information about the behaviour of 400 users across period of 1 months (30 days). Across those 30 days I measure 4 different information (let's call it A,B,C and D), hence I have a total of ...
3
votes
3answers
2k views

Are there any differences between Recurrent Neural Networks and Residual Neural Networks?

I'm currently studying the former and have heard of the latter, and right now I'm thinking that they're the same. Are they?
3
votes
4answers
770 views

Why is my test data accuracy higher than my training data?

I'm using four years of data, training on the first 3 and testing on the fourth. Using LSTM w/ Keras. My test data set (which has no overlap at all with the training) is consistently performing better ...
3
votes
3answers
1k views

Why are predictions from my LSTM Neural Network lagging behind true values?

I am running an LSTM neural network in R using the keras package, in an attempt to do time series prediction of Bitcoin. The issue I'm running into is that while my predicted values seem to be ...
3
votes
4answers
2k views

What is difference between feed forward neural network and LSTM?

What is the difference between feed-forward neural network and LSTM? How do they differ in their architecture?
3
votes
1answer
1k views

Stock prediction through LSTM

blue: training loss black: validation loss If modelling gives me such plot, what does it imply? Is it overfitting? How should I make it better? Code as below: ...
3
votes
2answers
3k views

loss/val_loss are decreasing but accuracies are the same in LSTM!

I am trying to train a LSTM model, but the problem is that the loss and val_loss are decreasing from 12 and 5 to less than 0.01, but the training set ...
3
votes
2answers
1k views

Unnormalized Log Probability - RNN

I am going through the deep learning book by Goodfellow. In the RNN section I am stuck with the following: RNN is defined like following: And the equations are : Now the $O^{(t)}$ above is ...
3
votes
3answers
3k views

My Keras bidirectional LSTM model is giving terrible predictions

I am trying to predict velocity (dynamics) values for notes that make up a piece of music using a bi-directional LSTM, following this blog post pretty closely: http://imanmalik.com/cs/2017/06/05/...
3
votes
2answers
185 views

Why can we not split train test data with 0.01 as parameter or 99% training data

Most of the blogs mention about a good thumb rule to be 80-20 split for the train and test respectively. My special case is a time series dataset and it is for the stock prices, which IMO is very ...
3
votes
3answers
131 views

Predict output sequence one at a time with feedback

I would like to solve the following classification problem: Given an input sequence and zero or more initial terms of the true output, predict the next term in the output sequence. For example, my ...
3
votes
1answer
2k views

in TensorFlow 2.0, what is the different between LSTM and LSTMCell objects?

I am trying to implement an RNN in TensorFlow 2.0 (beta1). Looking at the layer functions (inherited from Keras) I found: tf.keras.layers.LSTM and ...
3
votes
3answers
208 views

Using recurrent neural networks for modeling errors in IMUs

Inertial measurement units (IMU), usually composed of accelerometers and gyroscopes; are well known to have inherent errors in their data, originating from bias, random walk noise, temperature ...
3
votes
2answers
212 views

which NN should I use for Time-series dataset, whose pattern change as time goes

I am analyzing a time-series dataset using (supervised) tensorflow deep learning. The tensorflow code is given a series of inputs, and based on each input, the NN has to predict output value in near ...
3
votes
1answer
66 views

Recommender Model for Human Action in Income Protection

Problem Domain I'm working on a project that involves building a model to provide recommendations on the next best step for Human supervisors to take on income protection claims. Income protection is ...
3
votes
2answers
96 views

How is the Gaussian noise given to this BLSTM based GAN?

In a conditional GAN, we give a random noise along with a label to the generator as input. In this paper, I don't understand why in one section of the paper, they say they are giving the random noise ...
3
votes
1answer
845 views

LSTM: many to one and many to many in time-series prediction

I am trying to predict the trajectory of an object over time using LSTM. I have three different configurations of training and predicting values in my mind and I would like to know what the best ...
3
votes
1answer
30 views

Is it possible to create a neural network with two inputs, with sequential layers?

Is there a natural way, in terms of structure of the layers of a NN, in order to pass 2 inputs vectors to the NN? Example: text authorship identification Input #1: sentence1 by unknown author ...
3
votes
2answers
2k views

What is the vector value of [CLS] [SEP] tokens in BERT

In BERT, They replace separator and start of sentence with special token labels. What are there corresponding values in embedding_matrix. Are they 0-vector? I wanted to replace the proper nouns like ...
3
votes
2answers
128 views

training neural network

I was given the task as follows, Scrape articles appearing in Times of India since 2010 on HIV and AIDS. Classify them using training a neural network of your choice. Find patterns in those ...

1
2
3 4 5
18