The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

LSTM - Incorporate word embedding in layer with multiple records in same date

I have a time series data having more than one record in a single date. Number of records in a single date is not consistent. I have 3 input features namely phrase, cost and weight. My goal is to ...
0
votes
1answer
19 views

How can I explain the cause of different performances for two different LSTM models and improve the performance?

I've built two different models for Load Forecasting. Dataset has six features. The performance evaluation metric is the Mean Absolute Percentage Error(MAPE). Both models are based on LSTM. Here is ...
0
votes
1answer
22 views

LSTM loss function and backpropagation

I'm trying to understand the connection between loss function and backpropagation. From what I understood until now, backpropagation is used to get and update matrices and bias used in forward ...
0
votes
0answers
5 views

What do recurrent units like LSTM and GRU offer over global feedback?

Could anyone explain what LSTM, GRU and similar recurrent units offer over global feedback?
0
votes
2answers
58 views

Why LSTM models do not require labels for each step?

For time related problems like, for example, stock prediction: Let's say we have 300 days of data, 10 features, and one target: the price. Why, for the training, we only need the price of the 300th ...
1
vote
1answer
204 views

Metrics for presenting RNN/LSTM result

I am working on a two different architecture based on LSTM model to predict the users next action based on the previous actions. I am wondering, what is the best way to present the result? Is it okay ...
1
vote
1answer
34 views

Generate new sentences based on keywords

For example, for a domain specific neural network in Fashion, with the Keywords light, dress, orange, cotton. It could output: This gorgeous orange summer dress is great for wearing on sunny camping ...
0
votes
0answers
10 views

High Level Discussion: Generate synthetic sensor data using data from surrounding sensors

Let's assume I have readings coming from sensors. For every sensor, I have the following information: all the data it reads its location Now, given an arbitrary sensor at an arbitrary location ...
1
vote
1answer
20 views

Language modelling for Spell Checker

I am working on spell checkers, I want to create a spell checker, I am confused about which model to use Word-Level modelling Character-Level modelling plus I am preferring neural networks over ...
1
vote
1answer
21 views

Error on custom RNN/LSTM with multiple inputs

I want to implement a custom RNN/LSTM model similar to this. The model should take two separate vectors as input and process them. I was following keras tutorial to implement a custom keras layer and ...
5
votes
2answers
2k views

Relationship between batch size and the number of neurons in the input layer

Regarding LSTM neural networks, I am unable to understand the relationship between batch size, the number of neurons in the input layer and the number of "variables" or "columns" in the input. (...
1
vote
1answer
690 views

GRU/LSTM models - Train/Test split

I drove myself into a corner with this, can someone please explain? I feel I'm missing something obvious... If, for LSTM, each layer is trained with inputs from t and t-1, than that'd mean that if I'...
1
vote
0answers
21 views

Pytorch lstm model very high loss in eval mode against train mode

I am using a Siamese network with a 2-layer lstm encoder and dropout=0.5 to classify string similarity. For each batch, I am randomly generating similar and ...
1
vote
2answers
854 views
0
votes
0answers
9 views

Connection between Embedding and LSTM and Dense layer

I am building a "predict next word" model using the following model architecture. The codes fine, but I have a few questions: ...
1
vote
1answer
90 views

[Keras][LSTM] error due to shape mismatch

I have following data. Where I have 2 samples. Each sample I have 3 time steps each with 2 features. I intend to have 2 batches (to updates weights after every sample) ...
2
votes
1answer
49 views

Autoencoders for the compression of time series

I am trying to use autoencoder (simple, convolutional, LSTM) to compress time series. Here are the models I tried. Simple autoencoder: ...
0
votes
2answers
162 views

Keras/TF: Making sure image training data shape is accurate for Time Distributed CNN+LSTM

The comprehensible data shape to me is like: (9186, 120, 120, 1) this means 9186 entry, of 120 by 120 pixel grey images. I learnt that using Time Distributed to design a CNN combined with an LSTM ...
1
vote
1answer
72 views

How is PACF analysis output related to LSTM?

I was going through a recent paper “A Novel Hybrid Data-Driven Model for Daily Land Surface Temperature ForecastingUsing Long Short-Term Memory Neural Network Based on Ensemble Empirical ...
1
vote
1answer
584 views

Multi-Step Forecast for Multivariate Time Series (LSTM) Keras

I have been trying to understand how to build LSTM model for multivariate time series forecast using Keras but I am still unsure how to present the data in the correct shape. ...
0
votes
1answer
16 views

ReLU for combating the problem of vanishing gradient in RNN?

For solving the problem of vanishing gradients in feedforward neural networks, ReLU activation function can be used. When we talk about solving the vanishing gradient problem in RNN, we use a more ...
0
votes
2answers
442 views

TimeDistributed with different input / output sequence length

I'm looking into using TimeDistributed in my LSTM to see if it would improve the accuracy of my model. I'll be honest, I'm still not 100% sure what the specific ...
1
vote
1answer
378 views

How to design a many-to-many LSTM?

I have an input array of shape (1000,20, 4) and output(labels) of shape (1000,25,1). But don't know how to use Keras LSTM library to build a sequential model for this! Can someone help me design a ...
1
vote
2answers
789 views

LSTM future steps prediction with shifted y_train relatively to X_train

I'm trying to predict simple one feature time series data with shifted train data. The source looks like this: ...
-1
votes
2answers
40 views

LSTM to predict Sin(x) from x

Hi I want to pass a series of values x1, x2... as input to the model to predict y1 = sin(x1), y2 = sin(x2)... -I created dataset: x=[0.1,0.2,...] and y=[sin(0.1),sin(0.2),...] -I normalize x in [0,1]...
0
votes
1answer
20 views

How can I build a seq2seq model , which is topic aware

I have developed a chatbot, which is basically a seq2seq LSTM network. Which can generate text based on input text. But the problem I am having right now is it is not topic aware. As an example : ...
0
votes
2answers
47 views

How to normalize a data set of multiple time series?

I have the a data set representing the electricity consumption of 25 000 customer. The electricity readings are taken from each smart meter each 15 min for a period of 3 days. The data is takes from ...
2
votes
1answer
76 views

Running out of memory when training Keras LSTM model for binary classification on image sequences

I'm trying to come up with a Keras model based on LSTM layers that would do binary classification on image sequences. The input data has the following shape: ...
0
votes
1answer
21 views

The loss and accuracy of this LSTM both drop to nearly 0 at the same epoch

I'm trying to train an LSTM to predict the the Nth token using the N-1 tokens preceding it For each One-Hot encoded token, I ...
0
votes
0answers
15 views

Spatial and temporal information processing together (CNN and LSTM)

I have small problem that requires to process both spatial and temporal information. I need to predict vehicle's trajectory based on previous trajectory information and map information. My current ...
2
votes
2answers
2k views

Does this encoder-decoder LSTM make sense for time series sequence to sequence?

TASK given $\vec x = [x_{t=-3}, x_{t=-2}, x_{t=-1}, x_{t=0}]$ predict $\vec y = [x_{t=1}, x_{t=2}]$ Whith an LSTM encoder-decoder (seq2seq) MODEL NOTE: the ? symbol in the shape of the tensors ...
0
votes
1answer
15 views

Multivariate Multilag Regression with one shot prediction using LSTM

I am working on a multivariate regression task using a LSTM and I am interested in one shot prediction of my target variable (which is the price of a commodity). For example, the first parameter I ...
4
votes
1answer
79 views

Forecasting via LSTM or XGBoost… is it really a forecast or

I guess I understand the idea of predictions made via LSTM or XGBoost models, but want to reach out to the community to confirm my thoughts. This tutorial does a nice job explaining step by step of ...
1
vote
1answer
171 views

Using LSTM to predict binary classification - accuracy stuck at 50% - how to use statefulness

I am trying to use an LSTM model to make binary classifications; however when I train the model the loss stays around 0.69 (ie. -$\ln(0.5)$) and the accuracy at 0.5, which suggests to me the model is ...
1
vote
0answers
17 views

How can RNN handle variable sized inputs?

I came across this answer which is specific to Keras. But my question is at concept level. I am getting confused, How can RNN handle variable size inputs? here Let us suppose we want to do a ...
0
votes
1answer
23 views

Padding the sentences is consuming huge memory

I prepared a lstm model using tensorflow which has a max_sequence_length of 5000 and I'm padding the small sentences with 0's. I then deployed and tested the model ...
3
votes
2answers
58 views

Why can we not split train test data with 0.01 as parameter or 99% training data

Most of the blogs mention about a good thumb rule to be 80-20 split for the train and test respectively. My special case is a time series dataset and it is for the stock prices, which IMO is very ...
1
vote
2answers
52 views

Understanding Embeddings input and output sizes

I have been trying for a while to understand the dimensionality of embeddings in neural networks and I think that finally things have clicked on my brain, however I would love to check whether or not ...
3
votes
2answers
279 views

Is this a problem for a Seq2Seq model?

I'm struggling to find a tutorial/example which covers using an seq2seq model for sequential inputs other then text/translation. I have a multivariate dataset with n number of input variables each ...
1
vote
1answer
59 views

LSTM number of units for first layer

I'm trying to use LSTM (with Keras) for a time series problem. I would like predict the next value of the time series given its previous value. I'm using TimeseriesGenerator to create the training ...
0
votes
2answers
47 views

Using LSTM for binary text Classification, getting almost same accuracy at each epoch

I am doing Twitter sentiment classification. For that I am using LSTM with pretrained 50d GloVe word embeddings(not training them as of now, might do in future). The tweets are of variable lengths ...
0
votes
1answer
36 views

LSTM to multivariate sequence classification

How can I train multivariate to multiclass sequence using LSTM in keras? I have 50000 sequences, each in the length of 100 timepoints. At every time point, I have 3 features (So the width is 3). I ...
0
votes
0answers
12 views

How to get a output of a hidden layer of a single-layer LSTM

How can get the hidden layer outputs in a simple one-layer lstm? ...
0
votes
2answers
48 views

How to implement an LSTM RNN with multiple input features

EDIT: Now I didn't convert to list. I am training LSTM for multiple time-series in an array which has a structure: 450x801. There are 450 time series with each of 801 timesteps / time series. The ...
1
vote
1answer
18 views

Could the Input shape of the LSTM layer not be a constant?

I am trying a vanilla LSTM on my dataset. According to this course; the LSTM layer should be built as follows: ...
0
votes
1answer
21 views

How to represent the number of neurons in an LSTM for architecture schematic?

I'm trying to visualise a neural network schematic and found a great tool for building schematics here http://alexlenail.me/NN-SVG/index.html. I've edited the SVG file to change one of the dense ...
0
votes
0answers
17 views

LSTM forcasting: time series as input and a unique value for each as output

I am sorry for the mistakes I will make in this post, I'm new to deep learning ^^' I am trying to build an LSTM model that can help me predict some unique value according to time series indices and ...
2
votes
0answers
18 views

NLP based Data Preprocessing Method to Improve Disease Name Prediction Using CRF and Word Embedding

I built a model( using CRF along bi lstm) to Predict New Disease Name/Entities from medical text data but the problem is Disease name appears only 5,6 times in 1 text file but on average text file ...
1
vote
2answers
514 views

extraction information from resume

I have a project in machine learning in which I need to analyze a curriculum vitae. for that I have to write a python program. It uses basic techniques of Natural Language Processing like word ...