Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

Filter by
Sorted by
Tagged with
0
votes
1answer
54 views

Embedding layer before LSTM layer

I am toying around with a clustering and churn prediction framework, cluschurn which they deployed in production at Snap, Inc. In their research paper, paper_link, they use 14 days of user data and ...
0
votes
0answers
24 views

How does LSTM solve the vanishing gradient problem?

I know that there are many answers. shortly gates solve(mitigate) vanishing gradient problem. But I saw two formidable answers. Thomas Effland's answer, and Nir Abel's answer. I think they explain ...
1
vote
1answer
34 views

Making outputs of sub-models the inputs of other models LSTM

I'm trying to create a neural network that is composed of different sub-models. Each sub-model have their own inputs, which some of them can be shared between the sub-models. For example, sub-model 1 ...
0
votes
0answers
22 views

How to label sequential data for LSTM usage

I want to process data to feed it to a LSTM later, each 100 rows correspond to single category how should I label the data? Should I concatenate the 100 rows into a single row? Data Sample:- ...
1
vote
2answers
79 views

Language translation with convolutional neural network

Many examples of language translation neural networks: "the cat sat on the mat" -> [model] -> "le chat etait assis sur le tapis" use RNN, and in particular LSTM. See for example Sentences language ...
0
votes
1answer
43 views

How to feed training set labels into Keras LSTM

I’m implementing an LSTM with Keras and I know that I have to reshape the training dataset in a 3D object. Basically I have a ...
2
votes
1answer
27 views

Explanation for Why Logistic Regression can be so Accurate in Sentiment Classification?

My question is about how a logistic regression model performs so accurately. In some exploratory experimentation, I compared a logistic regression model against a long short term memory recurrent ...
0
votes
0answers
24 views

Backpropagation LSTM: compute gradient over multiple timesteps

Has anyone implemented simplified code to compute gradient of error over multiple timesteps for a single example. If the timesteps are large, even solving on paper it is getting really complicated. ...
0
votes
1answer
94 views

LSTM Multi-class classification for large number of classes

I want to build a model that classifies 473 classes -product categories-, but I'm facing a problem with loss not decreasing. Data I have almost 3,000 data points for each class -473 classes- (data ...
0
votes
0answers
30 views

LSTM backpropagation issue in Keras code

I have trained the network with data in batches of some batch size >1. After training, I am using trained network and then manually update parameters for every example using backpropagation. The ...
1
vote
1answer
34 views

Training on skewed dataset

I have a problem of multi class classification and I'm using a simple 2-Layer Bi-directional LSTM with keras. The model in a simple form: ...
-1
votes
1answer
34 views

LSTM for text with different sentences size, but same input-output sizes

Hello fellow Data Scientists I'm trying to use a LSTM (using word embeddings) to generate a system that can tag each word of a sentence. For this, I give it a set of sentences of different sizes and ...
1
vote
2answers
42 views

Feed data into Keras LSTM layer

I'm trying to understand how to feed data into LSTM layer of Keras, but I'm in trouble and I don't understand how to do it. I've ...
3
votes
1answer
28 views

Is it possible to create a neural network with two inputs, with sequential layers?

Is there a natural way, in terms of structure of the layers of a NN, in order to pass 2 inputs vectors to the NN? Example: text authorship identification Input #1: sentence1 by unknown author ...
0
votes
0answers
20 views

A training approach to do transfer learning on a pre-trained recurrent neural network to classify for each set of predicted features?

Let's say we want to classify the genre of a number of books as classes. We have a neural network that can read and encode each sentence in the book using a layer of neural network (e.g. word ...
2
votes
1answer
24 views

Forget Gate in Long Short-Term Memory

What value does actually LSTM forget in a training phase? for example, I do have a surface temperature data for 10 years. Then I made them as a training data for building my neural network using lSTM ...
1
vote
0answers
27 views

How can I get the predict future following value using Tensorflow LSTM?

Thank you for reading. I'm not good at English. I am wondering how to predict and get future time series data after model training. I would like to get the values after N steps. So, I used the time ...
0
votes
0answers
24 views

Dynamic sequence length in Keras LSTM layer

I'm implementing an LSTM with Keras to predict the correct words order. My dataset is composed by sentences, each sentences is ...
0
votes
0answers
22 views

LSTM autoencoder reconstructs input in ascending order

I implemented an autoencoder LSTM using Keras just as indicated in this article: article. The problem is that the reconstructed input of the time-series is given in ascending order with respect to the ...
2
votes
1answer
647 views

How to get feature importance from a keras deep learning model?

In case of scikit-learn's models, we can get feature importance using the relevant attributes of the model. I've been working on a RNN, using LSTMs for text embedding. Is there any way to get ...
1
vote
0answers
13 views

How can I do a sequence to sequence model (RNN / LSTM) with Keras with fixed length data?

What I'm trying to do seems so simple, but I can't find any examples online. First, I'm not working in language, so all of the embedding stuff adds needless ...
0
votes
0answers
15 views

Same Output for every row when doing prediction. Thus I am having wrong predictions

I have implemented an emotion analysis using lstm method, I have trained my model in the first part of the codes and i am now am doing the prediction part. When I run my system, I am having same ...
1
vote
1answer
23 views

How to return states of LSTM in MXNet?

I used MXNet previously to beat keras+tensorflow accuracy in CNN regression models. Now I am trying to implement LSTM, which in keras runs fine: ...
3
votes
2answers
78 views

Sentences language translation with neural network, with a simple layer structure (if possible sequential)

Context: Many language sentences translation systems (e.g. French to English) with neural networks use a seq2seq structure: "the cat sat on the mat" -> [Seq2Seq ...
0
votes
1answer
34 views

Understanding the concept vanishing gradient and exploding gradient problem in terms of training data

I'm trying to figure out the essence of the concepts "vanishing gradient and exploding gradient problem" in terms of real-world input-output training examples instead of in terms of the properties of ...
1
vote
0answers
23 views

How to setup LSTM problem when using multiple time series inputs?

Problem: Suppose I have 50 time series which I want to train on. For each series, given the last say 4 samples, I would like to predict the next value. Suppose each series is 100 samples. I break ...
1
vote
0answers
20 views

Time Series Forecasting with RNN/LSTM/NARX

I have some experimental datasets (like 4 or 5), and each dataset has three time series data, say $u1(t)$, $u2(t)$, and $x(t)$. The three time series of each experiment are similar but not the same. ...
2
votes
1answer
17 views

Use LSTM to predict the proportion of steps with nonzero feature values

I am trying to do a simple regression for sequences. Each input $X_i$ is a $n=2000$ by 1 matrix, formatted as $n_i$ 0-s followed by $(n-n_i)$ 1-s. The output $y_i$ should be $n_i/n$, i.e. the ...
0
votes
0answers
23 views

What does linearly decreasing training and accuracy loss means?

I am trying to train an LSTM autoencoder for sentence embeddings:- ...
0
votes
1answer
44 views

What is the output of multivariate LSTM model?

I am currently trying to build an LSTM model by using multivariate inputs, but I don't understand what exact output I am predicting. I am currently using 5 features in the data i.e. 'Time', 'Avg CPU ...
1
vote
1answer
243 views

How can I tune LSTM hyperparameters?

If anyone is there to answer these, that'll be great. I'm in the midst of a Final Year Project on LSTM. Currently, I’m stuck and confused over LSTM codes. There are 4 hyperparameters that I can play ...
0
votes
0answers
90 views

Root cause analysis using Machine Learning or NN

I am trying to build a predictive model that will analyze pattern from a snapshot file (log) and give the root cause analysis of the file i.e the class to which the pattern in the logs belongs to. I ...
2
votes
0answers
25 views

LONG SHORT-TERM MEMORY Hochreiter's paper BPTT

I was trying to read paper about LSTM, and I am stuck with mathematical problem. http://www.bioinf.jku.at/publications/older/2604.pdf page 4. see, |$f'_l$$_m$($net_l$$_m$)$w_l$$_m$ $_l$$_m$$_-$$_1$...
1
vote
1answer
84 views

Using GRU with FeedForward layers in Python

I'm trying to reproduce the codes in this paper here for the multi-labeling problem (11 classes), which is using ...
1
vote
0answers
33 views

Loss is decreasing correctly but upon prediction, totally wrong results

I've made a pretty basic stock prediction RNN with it's only input being the past stock price from apple. On this case, I want to input the apple stock price and the samsung stock price (2 features). ...
2
votes
1answer
385 views

Predicting parallel time series with multiple features

I am trying to predict sales for 2 departmental stores which share similar demographic properties. My goal is to make a single LSTM model to predict sales from these parallel time series having ...
2
votes
1answer
148 views

Can I use LSTM models to evaluate multiple, independent time series?

Let's say that I would like to predict the temperature tomorrow. I could use the approach whereby I train a model based on a time-series dataset collected from a single location (for example, see this ...
2
votes
1answer
39 views

Word Embedding or Hash?

In my dataset I have a 'text' column and a 'followers' column containing lists of follower IDs, i.e. '1093777852477116417, 936194589043683328,...'. Some of the 'followers' values contain thousands of ...
-1
votes
1answer
55 views

validation accuracy and loss increase

I am training a generic LSTM based autoencoder to get the sentence embeddings, the bleu score is the accuracy metric. The model is coded to output the same number of tokens as the length of labels, ...
2
votes
0answers
58 views

Help understanding the Tensorboard histogram names and meaning in an LSTM Model

Can someone please help me understand what the names and shapes of the following tensorboard histogram outputs mean about an LSTM model I coded? Thank you! I understand the terms in the names like ...
1
vote
0answers
97 views

TensorFlow 2: Find MAE, RMSE for validation dataset in time-series LSTM

TensorFlow 2 "Time series forecasting tutorial" (https://www.tensorflow.org/tutorials/structured_data/time_series#recurrent_neural_network) gives an example of a LSTM multi-step prediction model that ...
0
votes
1answer
56 views

Keras RNN (batch_size

I created RNN model for text classification with LSTM layer, but when I put the batch_size in the fit method, my model trained on the whole batch instead of just the mini batch _size. This also ...
1
vote
2answers
38 views

h in LSTM increasing in size?

So I was reading about the LSTM architecture and I was having trouble understanding a certain aspect of it. This article mentions the step in question near the bottom of the page. Here is the image ...
2
votes
2answers
96 views

What is the role of $W_{ax}, W_{aa}, W_{ay}$ in forward propagation in RNN? Are they hyperparameters? Why are they needed?

In RNN introduction in Coursera sequence model course, the following formula for forward propagation in RNN was introduced. What exactly is the role of $W_{ax}, W_{aa}, W_{ay}$? What do they do? In ...
1
vote
1answer
40 views

Using LSTMs for continous learning and predicting

I'm trying to develop a model to predict a commodity price movement direction based on previous observations. The model should learn common technical analysis patterns, e.g. head and shoulders. So, I ...
9
votes
1answer
2k views

Activation function between LSTM layers

I'm aware the LSTM cell uses both sigmoid and tanh activation functions internally, however when creating a stacked LSTM architecture does it make sense to pass their outputs through an activation ...
2
votes
0answers
49 views

Dense? or TimeDistributedDense? after LSTM layer in Keras

Dense and TimeDistributedDense, which one is suitable after LSTM layer in Keras? For example, ...
2
votes
0answers
39 views

Masking seems not working for missing values problem in LSTM

I am trying to use LSTM to predict time series in keras. My input data shape is (1000,6,1)(samples,timesteps,features). There is some missing data in different timesteps. For example,[2,1,1]=NaN,[3,4,...
3
votes
1answer
326 views

TensorFlow / Keras: What is stateful = True in LSTM layers?

Could you elaborate on this argument? I found the brief explanation from the docs unsatisfying: stateful: Boolean (default False). If True, the last state for each sample at index i in a batch will ...
2
votes
1answer
66 views

How to train Ner Model having 1 entity?

I am Creating a Custom NER (named entity recognition ) Model using bi directional LSTM and CRF. During Study on Ner i see all example includes Multiple entities per sentence. For eample this ...

1 2 3
4
5
17