Questions tagged [lstm]

LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.

Filter by
Sorted by
Tagged with
4
votes
3answers
369 views

Binary classification and numerical labels

I am trying to create a sentiment analysis model using a dataset that have ~50000 positive tweets that i labeled as 1, ~50000 negative tweets that i have labeled as 0. Also i have acquired ~10000 ...
1
vote
1answer
111 views

NER with LSTM - How to recognize person names that are not part of the vocabulary?

I am learning Named Entity Recognition and going through posts similar to this one: Named-Entity Recognition (NER) using Keras Bidirectional LSTM So the sentences are fed into the model as a sequence ...
2
votes
0answers
17 views

Predicting millions of sparse timeseries using them to help each other

This is a very general problem faced by different types of businesses. Predict the future behavior of customers over time. Imagine that we have 1 million customers with their own features over time, ...
0
votes
0answers
15 views

Regarding the application of RNN on new data

Suppose the two univariate time series $X_{1,T}=(x_1, x_2, ..., x_T)$ and $Y_{1,T}=(y_1, y_2, ..., y_T)$. The next step would be to train an RNN or LSTM with input $X_{1,T}$ and output $Y_{1,T}$, in ...
0
votes
0answers
307 views

Loss Nan: How can I properly implement a LSTM Time-Series model with a lot of parameters?

The Problem: I am very new to TF and Keras. I am attempting to train a time-series LSTM. When using only a few parameters as a test, the model seems to work fine. Once I increase the parameters to the ...
0
votes
0answers
48 views

How to use features with lags of different lengths in LSTM?

I'm trying to predict a time series, let's say I have 3 features and a target variable. I used the standard approach when feature lags have the same length, for example, 10. Then the size of my batch ...
0
votes
0answers
22 views

Derivation of HiddenState wrt Output of LSTM

I'm busy trying to understand the math behind LSTM RNN's. In most of the math tutorials that I've found the derivations (Backpropagation) don't consider a dense layer before the output, instead they ...
0
votes
0answers
49 views

Passing data to RNN with Sliding window approach

I having hard time with LSTM's and RNN so my apologies if this question sounds like a very basic question. I would appreciate if you can help in any way. I am trying to train my RNN with LSTMs, but I ...
1
vote
2answers
690 views

PyTorch: Predicting future values with LSTM

I'm currently working on building an LSTM model to forecast time-series data using PyTorch. I used lag features to pass the previous n steps as inputs to train the network. I split the data into three ...
1
vote
0answers
176 views

How to implement a Multivariate multi-site application in LSTM?

I am trying to make a multivariate multi-site classification LSTM model using Keras. I have followed this tutorial from Jason Brownlee: https://machinelearningmastery.com/multivariate-time-series-...
0
votes
0answers
38 views

Specifics of Input data format and processing in LSTM models

I'm busy writing a LSTM from scratch(just started) and I just want to clarify something. If I have a data shape as 1,10,2 which looks like: ...
0
votes
0answers
19 views

Batching data for LSTMs vs fully connected models

I've implemented an LSTM auto-encoder. It trained well, and does what I want it to so far. But, I think I've misunderstood something fundamental about lstms. In a simple dense network whose input ...
0
votes
0answers
90 views

Is my LSTM model overfitting or underfitting?

I am currently working on a project to classify comments text into 11 different topics, using a Bidirectional LSTM model. However, the loss curves confuse me as there is a deviation of the training ...
3
votes
2answers
211 views

Time-series multi-step generalization from single step model

I have built a generic stacked lstm model of the form: ...
2
votes
0answers
236 views

Keras model with LSTM quantization aware training

I would like to run quantization aware training with a keras model which has an LSTM layer. However, just the LSTM layer seems to not be supported. Alan Chiao seems to suggest here that it is possible ...
0
votes
1answer
21 views

How to detect time for the future events in time series data?

I am dealing with IOT data from a mechanical machine. On the input I have ~100 features that are measured every minute. On the output, I have labels of zeros and ones, where zero indicates the absence ...
0
votes
0answers
89 views

How to predict future prices with Keras LSTM time-series prediction model?

I have a trained and tested LSTM model which is meant to predict Ethereum close prices using all time csv data (24h steps). How do I now go about inputting an empty dataframe with future dates to ...
-1
votes
2answers
79 views

Stock Prediction Problem With LSTM [closed]

Lately I was working on a LSTM code to predict future stock prices. I did not get a good result from that. Also I saw a lot of articles talking about stock market data is a random walk and cant be ...
0
votes
0answers
66 views

My Stacked LSTM seems to be doing worse than a shallower one

I started with a two layer LSTM (+ Dense Layers) and which was: ...
1
vote
1answer
51 views

PyTorch: LSTM for time-series failing to learn

I'm currently working on building an LSTM network to forecast time-series data using PyTorch. I tried to share all the code pieces that I thought would be helpful, but please feel free to let me know ...
4
votes
1answer
1k views

Is a BiLSTM layer required if we use BERT?

I am new to Deep learning based NLP and I have a doubt - I am trying to build a NER model and I found some journals where people are relying on BERT-BiLSTM-CRF model for it. As far as I know BERT is a ...
1
vote
0answers
128 views

Concatenation of CNN and LSTM to model time of a series of images

I have collected a dataset consisting of around 30'000 heat maps of 80 users. The heat maps represent typing behavior on a keyboard and are just images with a resolution of ...
1
vote
1answer
120 views

Generate sentences from keywords by adding formal word

For example, I have a list of keywords like I, hungry => output: I am hungry or I, author, poem => output: I am the author of this poem. Can someone please suggest the simplest way to achieve ...
2
votes
1answer
56 views

Build a corpus for machine translation

I want to train an LSTM with attention for translation between French and a "rare" language. I say rare because it is an african language with less digital content, and especially databases ...
1
vote
1answer
85 views

Pytorch Luong global attention: what is the shape of the alignment vector supposed to be?

I am looking at the Luong paper on Attention models and global attention. I understand how the alignment vector is computed from a dot product of the encoder hidden state and the decoder hidden state. ...
0
votes
1answer
66 views

Predicting Bitcoin price with LSTM (and other RNNs)

In order to try what i learned about LSTM, i downloaded a simple bitcoin price dataset, and tried to make a network which predicts the bitcoin price by feeding a sequence of price history. Looking for ...
0
votes
1answer
49 views

Understanding projection layer for BLSTM

In many research papers there are 'projection layers' related to BLSTM layers. For example, from here: "we trained an 8-layer BLSTM encoder including 320 cells in each layer and direction, and ...
0
votes
0answers
26 views

Dropout after the Embeding layer

I am working on a classification problem. I am using pre-trained GloVe word embedding as input I wanted to know whether adding a dropout after the embedding layer makes sense at all? I have seen a lot ...
1
vote
0answers
129 views

strange behavior for validation dataset loss and accuracy

I'm training an LSTM model for a binary classification and using validation_split = 0.2 (in Keras), I see the following curves for the loss and accuracies of the train and test datasets. Here are two ...
0
votes
1answer
125 views

LSTM model prediction is almost constant

I am new to RNN and LSTM and currently experimenting with different settings. When trying to model time series data in absolute terms (predicted close price), I am faced with the following problems: ...
0
votes
1answer
24 views

Predicting Player Position from pervious positions and LSTMs

I am trying to use LSTMs to predict player positions in a field game. I try to overfit 8 slightly different time series. For this overfitting task I just use the positions of the players. A data ...
0
votes
1answer
20 views

Time series prediction with Lstm on patients data

My data includes different time series length (depends on number of exams each patient did) as well the interval between exams is different. How can I run lstm on this kind of data? (Where the ...
0
votes
1answer
29 views

Oscilations in loss curve [closed]

I saw a similar question, but I think my problem is something different. While training, the training loss and the validation loss move around one number, not decreasing significantly. I have 122707 ...
1
vote
1answer
85 views

LSTM with return_sequences - "Training a model on multiple timesteps simultaneously"

So I'm following Tensorflow's LSTM/time series tutorial and there's something I don't understand. I do understand what happens with return_sequences on/off, however,...
0
votes
1answer
146 views

Encoder-Decoder LSTM for Trajectory Prediction

I need to use encoder-decoder structure to predict 2D trajectories. As almost all available tutorials are related to NLP -with sparse vectors-, I couldn't be sure about how to adapt the solutions to a ...
1
vote
2answers
70 views

Choosing a model for input: categorised, weighted sequence, output: binary variable

What would be an appropriate model for predicting a binary target variable, given a weighted sequence? Sequences will be reasonably short, typically between ~ 1 and 5 elements. Illustrated example Say ...
1
vote
1answer
37 views

Long range forecasting with sequence-to-sequence models

I have a task where I want to forecast daily observations for 1 year or 2 years in advance at multiple locations--so 365 or 730 days in advance. I actually have a pretty good dataset, meaning daily ...
1
vote
0answers
162 views

LSTM giving almost constant output

I have used an LSTM with 4 layers deep each layer having 10 LSTM units to predict the AAPL stock 500 steps away by looking 50 steps back and it was predicting well (only a lag was there). However when ...
3
votes
1answer
97 views

Comparison between cost functions to determine the "best" model?

I'm building an LSTM neural net for time series prediction (regression) and I am incorporating custom loss functions into training. I'm trying to determine which cost function (of 3 cost functions) ...
1
vote
0answers
28 views

Simple LSTM model quickly learns and overfits

I am training a multi-class LSTM classifier on approximatively 700k documents of 40 words. My classes are very umbalanced, some have 2 or 3 samples while the biggest class has 48548 documents. My data ...
0
votes
1answer
63 views

LSTM BPTT with wide input?

I was following Tensorflow's own time series/LSTM tutorial, and there's something I don't quite understand about the whole process around Backpropagation Through Time (BPTT). The resources I've found ...
0
votes
1answer
74 views

Are there any control-flow/conditional statements in AI/ML models?

I was recently asked this during an interview. When we write a C program, it has a control-flow in the form of conditional statements like if, ...
2
votes
1answer
162 views

Why does the smallest LSTM I can make perform so well on this time series forecast?

So I've been playing with some different forecasting methods on a data set that I have done some more basic analyses for in the past. Without going into to much detail, it's population data over time ...
1
vote
1answer
34 views

How to score the health of a company? [closed]

i'm currently doing dual apprenticeships. My main mission is to represent the health of a company based on accounting records for multiple companies over multiple years. The part of an accounting ...
1
vote
2answers
33 views

Can I create a layer with multiple rnn cell ? [question about a paper]

I am trying to implement https://dl.acm.org/doi/pdf/10.1145/3269206.3271794 . Structure: As it said: In particular, we integrate the embedding vectors learned from each individual recurrent encoder ...
0
votes
0answers
26 views

How to reduce dimensionality of encoder decoder output?

I have an encoder decoder architecture where the output $ \bar{\bf{y}}_t $ is a sequence of integers of maximum length $n$. Each integer in the sequence is representative of a category so the ...
0
votes
0answers
16 views

Input variable that influences only trend

I need to predict cost for the next 4 weeks. Along with categorical variables (available for future as well), I have a numerical variable (a value from the day before prediction - I don't have future ...
1
vote
1answer
46 views

Working Behavior of BERT vs Transformers vs Self-Attention+LSTM vs Attention+LSTM on the scientific STEM data classification task?

So I just used BERT pre-trained with Focal Loss to classify Physics, Chemistry, Biology and Mathematics and got a good f-1 macro of 0.91. It is good given it only had to look for the tokens like ...
1
vote
0answers
304 views

What is the reason behind Keras choice of default (recurrent) activation functions in LSTM networks

Activation function between LSTM layers In the above link, the answer to the question whether activation function are required for LSTM layers was answered as follows: as an LSTM unit already consists ...
2
votes
1answer
4k views

ValueError: No gradients provided for any variable

I have this error when running training on my model. I found this issue on different sites, but could not find a solution to my problem. Here is my model : ...

1
3 4
5
6 7
21