Questions tagged [lstm]
LSTM stands for Long Short-Term Memory. When we use this term most of the time we refer to a recurrent neural network or a block (part) of a bigger network.
629
questions
1
vote
0answers
10 views
Pytorch lstm model very high loss in eval mode against train mode
I am using a Siamese network with a 2-layer lstm encoder and dropout=0.5 to classify string similarity.
For each batch, I am randomly generating similar and ...
0
votes
0answers
8 views
Connection between Embedding and LSTM and Dense layer
I am building a "predict next word" model using the following model architecture. The codes fine, but I have a few questions:
...
0
votes
1answer
14 views
ReLU for combating the problem of vanishing gradient in RNN?
For solving the problem of vanishing gradients in feedforward neural networks, ReLU activation function can be used.
When we talk about solving the vanishing gradient problem in RNN, we use a more ...
-1
votes
0answers
40 views
Predicting Distractor for QnA [closed]
Hello Everyone,
I need a help for a NLP task .
Problem Statement : The task is to build a model for QnA and predict it's most possible distractors.
Given: Below is a sample of train dataset
Train ...
-1
votes
2answers
40 views
LSTM to predict Sin(x) from x
Hi I want to pass a series of values x1, x2... as input to the model to predict y1 = sin(x1), y2 = sin(x2)...
-I created dataset: x=[0.1,0.2,...] and y=[sin(0.1),sin(0.2),...]
-I normalize x in [0,1]...
2
votes
1answer
67 views
Running out of memory when training Keras LSTM model for binary classification on image sequences
I'm trying to come up with a Keras model based on LSTM layers that would do binary classification on image sequences.
The input data has the following shape: ...
0
votes
1answer
19 views
The loss and accuracy of this LSTM both drop to nearly 0 at the same epoch
I'm trying to train an LSTM to predict the the Nth token using the N-1 tokens preceding it
For each One-Hot encoded token, I ...
0
votes
0answers
13 views
Spatial and temporal information processing together (CNN and LSTM)
I have small problem that requires to process both spatial and temporal information.
I need to predict vehicle's trajectory based on previous trajectory information and map information. My current ...
2
votes
1answer
46 views
Autoencoders for the compression of time series
I am trying to use autoencoder (simple, convolutional, LSTM) to compress time series.
Here are the models I tried.
Simple autoencoder:
...
0
votes
1answer
18 views
How can I build a seq2seq model , which is topic aware
I have developed a chatbot, which is basically a seq2seq LSTM network. Which can generate text based on input text. But the problem I am having right now is it is not topic aware.
As an example :
...
4
votes
1answer
76 views
Forecasting via LSTM or XGBoost… is it really a forecast or
I guess I understand the idea of predictions made via LSTM or XGBoost models, but want to reach out to the community to confirm my thoughts.
This tutorial does a nice job explaining step by step of ...
1
vote
0answers
16 views
How can RNN handle variable sized inputs?
I came across this answer which is specific to Keras. But my question is at concept level. I am getting confused, How can RNN handle variable size inputs?
here
Let us suppose we want to do a ...
0
votes
1answer
22 views
Padding the sentences is consuming huge memory
I prepared a lstm model using tensorflow which has a max_sequence_length of 5000 and I'm padding the small sentences with 0's. I then deployed and tested the model ...
0
votes
2answers
47 views
How to normalize a data set of multiple time series?
I have the a data set representing the electricity consumption of 25 000 customer.
The electricity readings are taken from each smart meter each 15 min for a period of 3 days. The data is takes from ...
3
votes
2answers
54 views
Why can we not split train test data with 0.01 as parameter or 99% training data
Most of the blogs mention about a good thumb rule to be 80-20 split for the train and test respectively.
My special case is a time series dataset and it is for the stock prices, which IMO is very ...
0
votes
2answers
47 views
Using LSTM for binary text Classification, getting almost same accuracy at each epoch
I am doing Twitter sentiment classification. For that I am using LSTM with pretrained 50d GloVe word embeddings(not training them as of now, might do in future).
The tweets are of variable lengths ...
0
votes
0answers
12 views
How to get a output of a hidden layer of a single-layer LSTM
How can get the hidden layer outputs in a simple one-layer lstm?
...
1
vote
1answer
18 views
Could the Input shape of the LSTM layer not be a constant?
I am trying a vanilla LSTM on my dataset.
According to this course; the LSTM layer should be built as follows:
...
0
votes
0answers
17 views
LSTM forcasting: time series as input and a unique value for each as output
I am sorry for the mistakes I will make in this post, I'm new to deep learning ^^'
I am trying to build an LSTM model that can help me predict some unique value according to time series indices and ...
1
vote
1answer
33 views
Generate new sentences based on keywords
For example, for a domain specific neural network in Fashion, with the Keywords light, dress, orange, cotton. It could output: This gorgeous orange summer dress is great for wearing on sunny camping ...
2
votes
0answers
18 views
NLP based Data Preprocessing Method to Improve Disease Name Prediction Using CRF and Word Embedding
I built a model( using CRF along bi lstm) to Predict New Disease Name/Entities from medical text data but the problem is Disease name appears only 5,6 times in 1 text file but on average text file ...
0
votes
0answers
23 views
pass tf.data.Dataset to input layer in tf.keras functional api in LSTM models
While trying to use an Input layer prior to embedding layer in Keras functional design model, the Batch dimensions are not ignored by input layer automatically.
Here is my ...
0
votes
1answer
31 views
LSTM to multivariate sequence classification
How can I train multivariate to multiclass sequence using LSTM in keras?
I have 50000 sequences, each in the length of 100 timepoints.
At every time point, I have 3 features (So the width is 3).
I ...
0
votes
0answers
27 views
How to use TimeDistributed fo CNN+LSTM?
I am trying to classify 6 classes time-frequency domain signal (STFT spectrogram) with a size of 3601x217 pixels. Assume that for each classes have 70 training samples, 20 validation samples, and 10 ...
0
votes
0answers
7 views
Loss and accuracy remains constant in time series classification by LSTM
I have a time series data with a classification label of 1 and 0. I am using a LSTM model to classify the series by taking 100 consecutive timestamps as input with a single label.
Even after training ...
3
votes
1answer
34 views
Do timesteps must have the same temporal distance in training a RNN?
I have a recurrent neural network with LSTM units that I want to train with batches of 6 timesteps. Each timestep is a record of a dataset and represents the temporal aggregation over 5 minutes of ...
1
vote
1answer
68 views
2D-Input to LSTM in Keras
I have following problem: I would like to feed LSTM with
train_datagen.flow_from_directory
The input is basically a spectrogram images converted from time-series into time-frequency-domain in PNG ...
0
votes
0answers
5 views
To train in Keras two identical RNN but with different outputs
I am training to solve a problem where I am using a RNN to track an object, learn about it, and then generate trajectories. Therefore the input of the RNN is stuff like x,y,speed,... and the output is ...
1
vote
0answers
35 views
Why is LSTM implemented in tensorflow so slow as compared to pure pythonic implementation
I am trying to implement word level prediction, an adaptation of from http://karpathy.github.io/2015/05/21/rnn-effectiveness/. When I implement it in pure python, the training is fast. However I ...
0
votes
1answer
19 views
LSTM loss function and backpropagation
I'm trying to understand the connection between loss function and backpropagation.
From what I understood until now, backpropagation is used to get and update matrices and bias used in forward ...
0
votes
0answers
14 views
Pytorch Implementing Simple Attention using Dummy data
Hi I am trying to implement simple/General attention in Pytorch , So far the model seems to working , but what i am intersted in doing is getting the attention weights , so that i can visualize it .
...
0
votes
1answer
30 views
How to convert sequence of words in to numbers which are input to RNN/LSTM?
I am watching online videos and tutorials about use of RNN/LSTM for NLP but none of them explain how to convert the sequences of words into digitized input to the neural networks?
I am looking for ...
0
votes
0answers
11 views
Day Classification in Time Series - LSTM
I am working on a problem in which i have a daily time series and I have a label for each day. For simplicity, let's say it is a binary classification, so for each day, there's a label (0 or 1) and 1 ...
0
votes
0answers
19 views
processing sequence of sequences in PyTorch
I try to deal with some special sequences via recurrent modules and I have faced some non-trivial things:
Sequences are not in same length.
I have sequence of sequences, and my idea is to encode ...
0
votes
0answers
33 views
LSTM architecture for multivariate time series
For a multivariate time series analysis, which of the following LSTM architectures would work better and why?
1) Having two independent LSTM layers (one for the time series variable and one for the ...
0
votes
0answers
25 views
CNN Combinined with LSTM
I tried to combine CNN with LSTM for depression detection using the following code
...
2
votes
0answers
14 views
Simmultainiously calculting loss from target interdependend metric
Is there a way to incorporate multiple targets into one loss?
Currently, I work with the Sequential() API, I guess this won't be sufficient....
I work with area predictions as targets. Each sample ...
0
votes
1answer
13 views
Saving LSTM hidden states while training and predicting for multi-class time series classification
I am trying to use an LSTM for multi-class classification of time series data.
The training set has dimensions (390, 179), i.e. 390 objects with 179 time steps each.
There are 37 possible classes.
...
0
votes
0answers
18 views
How to use LSTM for time series data?
I've an ECG data spread over time. The duration for each data is around 3 minutes (approx 180 seconds).
Each second around 200 recordings were taken. So total length for each sample is approx 36000.
...
1
vote
1answer
54 views
LSTM number of units for first layer
I'm trying to use LSTM (with Keras) for a time series problem. I would like predict the next value of the time series given its previous value. I'm using TimeseriesGenerator to create the training ...
2
votes
0answers
29 views
What is the current state-of-the-art video classification technique?
For a project I am aiming to automise the detection of goals in foosball (a.k.a. 'table football') matches. To do so I now track the ball in every frame using the openCV library in Python.
To ...
2
votes
0answers
34 views
Structuring a LSTM Layer
I'm trying to improve an NER Bert sequence tagger using LSTM layers in TensorFlow. I'm a bit unclear on the interface and how a LSTM layer should be set up.
Currently, I'm taking in 3-5 sentences and ...
0
votes
0answers
16 views
Framing Multiple Parallel Series problem for LSTM
I'm dealing with a multiple parallel time series problem (see here) and I have an issue regarding the non-constant number of sample per timestamp (not timestep!)
My problem is the following:
I have ...
0
votes
1answer
20 views
How to represent the number of neurons in an LSTM for architecture schematic?
I'm trying to visualise a neural network schematic and found a great tool for building schematics here http://alexlenail.me/NN-SVG/index.html. I've edited the SVG file to change one of the dense ...
0
votes
2answers
46 views
How to implement an LSTM RNN with multiple input features
EDIT: Now I didn't convert to list.
I am training LSTM for multiple time-series in an array which has a structure: 450x801. There are 450 time series with each of 801 timesteps / time series. The ...
0
votes
0answers
13 views
Changing data structure in incremental learning of LSTM
This is a question which may or may not have open-ended answers. I am curious what you think and hoping to get a starting point. I am wondering what we do if we have a categorical variable in the set, ...
0
votes
0answers
18 views
Categorical Multivariate Time Series
I have a small dataset of products of which the price varies along time.
Each product is represented by categorical features mostly ( type, matter, use, location ...) and one or two scalar features ( ...
0
votes
0answers
39 views
How multi layer LSTM are interconnected?
I am trying to understand the layers in LSTM for my own implementation using Python. I started with Keras to getting familiarized with the layer flow.
I have tried the below code in Keras and I have ...
2
votes
0answers
18 views
Why might an LSTM be capable of predicting an ARMA signal but not a linear combination of ARMA signals?
I have an LSTM network and am testing it on some dummy ARMA signals. I'm trying to predict the signal 5 time steps into the future.
The network is capable of outperforming Naive (persistence) when ...
0
votes
0answers
6 views
How is possible the result of GRU would other way around compared to LSTM while they were implemented samely?
Recently I crossed to a situation I can't figure it out why it happened. I applied six predictive models on a specific dataset as training-set and tried to predict the other similar dataset as an ...
