Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

Filter by
Sorted by
Tagged with
2
votes
1answer
71 views

Philosophical question on redundancy

Suppose I implement a supervised learning version of LSTM similar to this. Namely, I have these univariate time series data: ...
2
votes
1answer
43 views

Papers on anger detection in dialogues

I am interested in anger detection in dialogues and I want to study multiple methods like LSTM, CNN, etc. Are there any good research papers or books about this subject?
2
votes
1answer
403 views

Which scaling size is better? [0,1] or [-1,1] for LSTM?

I see some scale their data between 0 and 1 and some others do that between -1 and 1. But which one is better? Or better to ask: How to make a true/good decision for that?
2
votes
1answer
2k views

How to reshape data for LSTM training in multivariate sequence prediction

I want to build an LSTM model for customer behaviour. It's the first time for me working on a timeseries, so some concepts are not clear to me at all. My prediction problem is multidimensional, ...
2
votes
1answer
744 views

How to optimally train deep learning model using output as new input

I'm trying to train a network to predict the future. My current setup uses 5 time steps as inputs from the past, each consisting of 10 features, resulting in a [5, 10] input matrix (initially ...
2
votes
1answer
1k views

Using SMAPE as a loss function for an LSTM

I am currently working on a time series forecasting problem and am looking into using an LSTM. My final accuracy metric that I use to determine whether or not the forecast is good or not is defined ...
2
votes
2answers
684 views

What is the relation between input into LSTM and number of cells?

I want to train an LSTM network for time-series predictions, and want to get to the bottom of LSTM's. In my understanding, the number of cells in a single LSTM layer can vary. However, since each cell ...
2
votes
1answer
157 views

Structure of a multilayered LSTM neural network?

I implemented a LSTM neural network model in Keras. However, how the codes worked under the hood was not quite clear. I want to know if it worked the way I guessed how it worked? For example: Say ...
2
votes
1answer
792 views

What does GlobalMaxPooling1D() do to output of LSTM unit in Keras?

The keras model looks like this ...
2
votes
1answer
2k views

How to choose dimensionality of the Dense layer in LSTM?

I have a task of multi-label text classification. My dataset has 1369 classes: ...
2
votes
3answers
2k views

What should be 'y_train' in Keras LSTM?

I refer to the example given at the Keras website here: ...
2
votes
1answer
448 views

Fine-tuning NLP models

In computer vision, if we don't have a large training set, a common method is to start with a pre-trained model for some related task (e.g., ImageNet) and fine-tune that model to solve our problem. ...
2
votes
1answer
2k views

Very long sequence in neural networks

Beginner's question regarding sequences in neural networks: suppose I have classification problem that looks like: X = very long sequence of varying length. Y = class (assume for simplicity y=0/1). ...
2
votes
2answers
2k views

Dropout Decreases Test and Train Accuracy in one layer LSTM in Pytorch

I have a one layer lstm with pytorch on Mnist data. I know that for one layer lstm dropout option for lstm in pytorch does not operate. So, I have added a drop out at the beginning of second layer ...
2
votes
2answers
711 views

LSTM predicting expected value and standard deviation with one-hot encoding

I am implementing LSTM to a time series prediction which requires predicting both the expected value (i.e. the mean of the prediction), and the standard deviation (i.e. the interval that the future ...
2
votes
1answer
675 views

LSTM Validation MSE always lower than Train MSE

I am trying to train a LSTM network to forecast time steps further. I have a list of queries and the current question is based on one among them. The validation loss (using mse) is always lower than ...
2
votes
2answers
7k views

LSTM - How many times should I look back to predict next six hours -Multivariate Time-Series

I am still finding confusing on look back topic when using LSTM for time-series analysis. If I have hourly data and I want to predict next 6 hours with multiple ...
2
votes
2answers
1k views

LSTM: Taking previous output values as feature

As far as I know, there is practically no limit on the number of dimensions of input feature for LSTM. And it apparently can learn the sequence of data. My question is does LSTM by nature, also take ...
2
votes
2answers
39 views

Extracting location from text - NOT sensetive to letters (Upper or Lower Case) or already known vocabulary words

I would like to extract location or contents related to location from raw text. I used the NLTK and spaCy packages already; none worked for me. For example, both would neglect 'canada' as a location ...
2
votes
1answer
174 views

How to present longitudinal data to LSTM for multiclass prediction

I need to implement a deep learning algorithm to predict an ordinal value, called 'Entity', using longitudinal health records data. I read a few articles and guides but I couldn't find a clear ...
2
votes
1answer
56 views

Build a corpus for machine translation

I want to train an LSTM with attention for translation between French and a "rare" language. I say rare because it is an african language with less digital content, and especially databases ...
2
votes
1answer
4k views

ValueError: No gradients provided for any variable

I have this error when running training on my model. I found this issue on different sites, but could not find a solution to my problem. Here is my model : ...
2
votes
1answer
1k views

LSTM followed by Dense Layer in Keras

I am working on LSTMs and LSTM AutoEncoders, trying different types of architectures for multivariate time series data, using Keras. Since it is not really practical to use relu in LSTM because of ...
2
votes
1answer
27 views

Does the output of the Sequence-to-Sequence encoder model exist in the same semantic space as the inputs (Word2vec)? [closed]

Does the output generated from the LSTM encoder module exist in the same semantic space as the original word vectors? If so, say for example we have a sentence and we pass it through the encoder to ...
2
votes
1answer
29 views

GRU and LSTM does not "take risk" predicting

I tested LSTM and GRU models to predict the exchange rate between currencies. I do not take the raw price but a the delta with the previous day, so the data is stationnary around zero. My problem is ...
2
votes
2answers
435 views

What are suitable datasets for univariate time series forecasting with RNNs, LGBM, TBATS, SARIMA models (topic, frequency, sources)? [closed]

I am currently looking for a suitable dataset (univariate time series) for short-term forecasting using lag features or moving windows of lag features to employ models like LSTM, GRU, SARIMA, LGBM, ...
2
votes
2answers
486 views

How similar is Adam optimization and Gradient clipping?

According to the Adam optimization update rule: $$m \leftarrow \beta_1 m + (1 - \beta_1)\nabla J(\theta)$$ $$v \leftarrow \beta_2 v + (1 - \beta_2)(\nabla J(\theta) \odot \nabla J(\theta))$$ $$\theta \...
2
votes
1answer
107 views

static and dynamic data in clinical trials

Hi everybody and thanks in advance for those who will help me for this problem. I have multiple data regarding patients involved in a clinical trial and my goal is to predict their death/non death. ...
2
votes
1answer
47 views

How do you think about neural networks and ways to design new models?

I'm currently learning about neural networks, and it seems to me that there usually is no good theoretical explanation given for why certain architectures work; there is most of the times, no formal ...
2
votes
1answer
81 views

Forget Gate in Long Short-Term Memory

What value does actually LSTM forget in a training phase? for example, I do have a surface temperature data for 10 years. Then I made them as a training data for building my neural network using lSTM ...
2
votes
1answer
21 views

Use LSTM to predict the proportion of steps with nonzero feature values

I am trying to do a simple regression for sequences. Each input $X_i$ is a $n=2000$ by 1 matrix, formatted as $n_i$ 0-s followed by $(n-n_i)$ 1-s. The output $y_i$ should be $n_i/n$, i.e. the ...
2
votes
2answers
323 views

Dense? or TimeDistributedDense? after LSTM layer in Keras

Dense and TimeDistributedDense, which one is suitable after LSTM layer in Keras? For example, ...
2
votes
1answer
693 views

building a 2-layer LSTM for time series prediction using tensorflow

From Tensorflow tutorials i am experimenting time series with LSTM In the section 'multi-step prediction' using LSTM tutorial says Since the task here is a bit more complicated than the previous ...
2
votes
2answers
1k views

Training LSTM for time series prediction with nan labels

I have a time series of features $x_1,x_2,x_3,...,x_n$. I want to make a prediction $y_1,y_2,y_3,...,y_n$ for each timestep. However, in my training data some of the $y$ can be nan. I'd like the fit ...
2
votes
1answer
195 views

transform a supervised neural network to reinforcement learning?

I have a functional LSTM model that works with an acceptable performance. How can I now convert this supervised model to a reinforcement learning model for improving the performane? Is there any ...
2
votes
1answer
88 views

NLP - Identify Tagged Words

Please pardon me as the title might not be very accurate I am trying to create a model that learns the word representation and then is able to predict word representation in another piece of text. An ...
2
votes
1answer
892 views

MinMaxScaler when LSTM predictions fall outside of training range?

I am using MinMaxScaler on my training set and applying the transformations to my test set and inverse_transform to my model’s outputs. If this were, say, a stock prediction problem, my training set ...
2
votes
1answer
1k views

Time Series Forecasting for Multiple Customers using one RNN

I have a product which has univariate and also multivariate time series data from multiple customers. I have variable amount of data available. Ranging between couple of years to couple of months. ...
2
votes
1answer
259 views

Best way to classify plots which are overlapping?

I have an experiment in which it was done under two conditions. For each condition, the experiment was performed 26 times. The output of the experiment is a plot with 70 time indices. I would like to ...
2
votes
1answer
125 views

Working ofLSTM with multiple Units - NER

I am trying to understand working of LSTM networks and kind of not clear about how different neurons in a cell interact each other. I had a look at a similar question, but still not clear about few ...
2
votes
1answer
43 views

Why do recurrent layers work better than simple feed-forward networks?

On a time series problem that we try to solve using RNNs, the input usually has the shape $input features \times timesteps \times batchsize$ and we then feed this ...
2
votes
1answer
105 views

How can I specify I want output of which units in LSTM?

I see this code concept(with Keras library) in most code examples of LSTM: model.add(LSTM(X)) model.add(Dense(Y)) But I don't ...
2
votes
1answer
2k views

LSTM Time series prediction for multiple multivariate series

I have to predict next min traffic for multiple cities (100+). I am thinking of using LSTM. My main concern is how do I scale the number of cities. How does LSTM learn different amount of traffic and ...
2
votes
3answers
158 views

Which neural network to choose for classification from text/speech?

I am considering two tasks: Dialog Act Classification from Text (e.g. classify to: question; opinion; ...) Emotion Recognition from Speech (e.g. happy; calm; sad; ...) Which DL model should perform ...
2
votes
1answer
857 views

Give Variable Length input to LSTM

My input data consist of list of list. Both list have dynamic length for every example like below. ...
2
votes
2answers
2k views

Model Not Learning with Sparse Dataset (LSTM with Keras)

This classification problem is apparently simple and I have no idea why it's not working, perhaps I'm doing a conceptual mistake. I'm trying to make a predictor which will classify minutes on a clock ...
2
votes
1answer
192 views

How to implement keras LSTM time series [closed]

I am learning how to implement Keras LSTM on a simple time series data. The dataset I'm using has $12$ columns and $300k$ rows. Each group of $200$ rows represents ...
2
votes
1answer
2k views

Shaping data for ConvLSTM for many-to-one image model

Ultimately, I am trying to obtain a binary segmentation mask for an image sequence. I have n number of image sequences, each with 500 greyscale images of size 256px by 400px. Each of these sequences ...
2
votes
1answer
256 views

How many RNN units are needed for tasks involving sequences?

I am training an RNN on the following task: Given a sequence of thirty words, predict the next word. Is there a benefit to having more than 30 cells (LSTM, GRU or plain RNN) in my network? I've seen ...
2
votes
1answer
2k views

Recurrent Neural Net (LSTM) batch size and input

I am working in Keras to build LSTM models. I understand that setting STATEFUL=FALSE means that the different batches are treated as independent when training the model. Suppose I want to build a ...

1
3 4
5
6 7
21