All Questions
37 questions
0
votes
0
answers
28
views
GPU running out of memory for ConvLSTM model in Pytorch
I'm trying to replicate the bidirectional convolutional LSTM proposed in Xiong et al. 2017 to predict crowd count density maps, but I'm running into memory issues during the training. This is what I'...
- 11
0
votes
0
answers
17
views
Adding sliding window dimension to data causes error: "Expected 3D or 4D (batch mode) tensor ..."
I wrote a pytorch data loader which used to return data of shape (4,1,192,320) representing the 4 samples of single channel image, each of size ...
- 219
0
votes
0
answers
40
views
PyTorch input shape for text classification using LSTM
I have three sentiment classes: POSITIVE, NEGATIVE, and NEUTRAL, along with a dataset consisting of 3000 sentences and their corresponding sentiment labels (POSITIVE, NEGATIVE, or NEUTRAL). Each ...
0
votes
2
answers
650
views
How to add multiple embeddings (layers) to LSTM layer
The similar question was asked before here https://stackoverflow.com/questions/52627739/how-to-merge-numerical-and-embedding-sequential-models-to-treat-categories-in-rn/52629902#...
1
vote
1
answer
3k
views
Is sequence length and hidden size is the same in Transformer
I'm confused about sequence length and hidden size if they are the same in Transformer.
I think they are different but not sure.
...
- 127
1
vote
0
answers
152
views
Backward propagation slower as epochs increase in pytorch
I'm trying to replicate TasNet paper using Pytorch Lightning, but the training time increases as the epochs increase (the first epoch takes 20 seconds, the fifth 2:30 minutes).
I've used PyTorch ...
0
votes
1
answer
589
views
PyTorch mat1 and mat2 shapes cannot be multiplied (100x200 and 100x9922)
I am trying to make a BiLSTM language model and am having some issues.
Model
...
1
vote
1
answer
539
views
pytorchs LSTMs use of 'bias' and 'weight' strings
Hi I am new to RNN and have come across this the following implementation of Pytorchs LSTM, but I cant understand how (or why) the 'bias' and ...
- 135
1
vote
1
answer
176
views
In LSTM why h_t output twice?
According the LSTM design:
The hidden state (ht) is output twice (1 and 2 in the picture).
If they are the same, why we need them twice ?
Is there a different use for each one of them ?
According to
...
- 527
2
votes
1
answer
3k
views
What is the purpose of Sequence Length parameter in RNN (specifically on PyTorch)?
I am trying to understand RNN. I got a good sense of how it works on theory. But then on PyTorch you have two extra dimensions to your input data: batch size (number of batches) and sequence length. ...
- 23
1
vote
0
answers
753
views
LSTM Text Generation with Pytorch
I am currently trying quote generation (character level) with LSTMs using Pytorch. I am currently facing some issues understanding exactly how the hidden state is implemented in Pytorch.
Some details:
...
0
votes
1
answer
908
views
Which output of a BiLSTM layer should be used for classification
I am trying to implement a BiLSTM layer for a text classification problem and using PyTorch for this.
...
1
vote
0
answers
147
views
Which One is the Best Way to Create Training Sequences for LSTM-based Class Prediction on Time-series Data?
Let's say I have time-series data in the following way. I need to create training sequences of a fixed length as an input to my LSTM model on PyTorch.
...
- 171
0
votes
1
answer
347
views
How to pass a sequence of 4 images into LSTM and CNN-LSTM
I got an assignment and stuck with it while going down the rabbit hole of learning PyTorch, LSTM, and CNN.
Provided the well-known MNIST library I take combinations of 4 numbers and per combination, ...
- 33
2
votes
0
answers
87
views
Time series forecasting. How use future values
I have a time series dataset containing hourly data from a few year, like below. Let's assume that I want to make prediction for the next 3 hours (2021-01-01 19:00, 2021-01-01 20:00, 2021-01-01 21:00)....
- 21
0
votes
1
answer
745
views
Help improving time series prediction with LSTM on PyTorch
So, I am trying to use a LSTM model to forecast temperature data on PyTorch. I am relatively new to both PyTorch and the use of recurrent networks so I took a model I found on the internet to start. ...
- 71
1
vote
2
answers
2k
views
PyTorch: Predicting future values with LSTM
I'm currently working on building an LSTM model to forecast time-series data using PyTorch. I used lag features to pass the previous n steps as inputs to train the network. I split the data into three ...
- 151
0
votes
0
answers
31
views
Batching data for LSTMs vs fully connected models
I've implemented an LSTM auto-encoder. It trained well, and does what I want it to so far. But, I think I've misunderstood something fundamental about lstms.
In a simple dense network whose input ...
- 309
1
vote
1
answer
858
views
PyTorch: LSTM for time-series failing to learn
I'm currently working on building an LSTM network to forecast time-series data using PyTorch. I tried to share all the code pieces that I thought would be helpful, but please feel free to let me know ...
- 151
1
vote
1
answer
206
views
Pytorch Luong global attention: what is the shape of the alignment vector supposed to be?
I am looking at the Luong paper on Attention models and global attention. I understand how the alignment vector is computed from a dot product of the encoder hidden state and the decoder hidden state. ...
- 173
1
vote
1
answer
1k
views
Pytorch LSTM not training
So I am currently trying to implement an LSTM on Pytorch, but for some reason the loss is not decreasing. Here is my network:
...
user98969
1
vote
1
answer
495
views
LSTM model prediction scaling with loaded model
I am deploying a LSTM pytorch model for production and I have issue with scaling the LSTM output correctly. While the model was tested the output was scaled with label data:
...
- 111
0
votes
1
answer
81
views
LSTM / GRU weights during test time
I am working on a historic time series dataset and using RNN, LSTM, GRU models, and I didn't find an answer if in test time, the h (or h, c) weights should be zeors for each batch?
If the weights ...
- 11
1
vote
0
answers
30
views
Learning the distribution of a continuous variable using LSTM
I am trying to implement the following paper : https://arxiv.org/pdf/2006.10701.pdf. In order, to estimate the priors of the hidden states which have continuous values, the authors use a LSTM. I have ...
- 11
1
vote
0
answers
62
views
Text generation - Input text (one sentence or many sentences)
I am currently working on a project: I want to generate text with a LSTM using Pytorch. My model is working but I have a question about the methodology:
I'm using the BPTTIterator and something seems ...
- 11
-1
votes
1
answer
204
views
Bidirectional vs. Traditional LSTM [closed]
I'm working on image captioning problem, where I need to have an encoder for image and decoder for caption generation. Regarding the decoder, I've found a reference that uses Pytorch LSTM where ...
2
votes
1
answer
267
views
Predicting sequence element based on the previous M and the following N elements
I have an array of sequences of equal length, each sequence contains 300 numbers (M=300). Each element in a sequence is a number from 1 to 9:
...
- 85
0
votes
1
answer
176
views
Considering the output of a BLSTM in pytorch, what's the order of the elements?
I am currently using pytorch to implement a BLSTM-based neural network. I understand that the output of the BLSTM is two times the hidden size. However, I am currently unable to find out whether this ...
- 13
0
votes
1
answer
1k
views
LSTM Multi-class classification for large number of classes
I want to build a model that classifies 473 classes -product categories-, but I'm facing a problem with loss not decreasing.
Data
I have almost 3,000 data points for each class -473 classes- (data ...
- 103
1
vote
1
answer
5k
views
How to calculate perplexity in PyTorch?
I am wondering the calculation of perplexity of a language model which is based on ...
- 155
1
vote
0
answers
384
views
Pytorch lstm model very high loss in eval mode against train mode
I am using a Siamese network with a 2-layer lstm encoder and dropout=0.5 to classify string similarity.
For each batch, I am randomly generating similar and ...
- 11
3
votes
0
answers
310
views
Why embedding or rnn/lstm can not handle variable length sequence?
Pytorch embedding or lstm (I don't know about other dnn libraries) can not handle variable-length sequence by default. I am seeing various hacks to handle variable length. But my question is, why this ...
- 521
2
votes
0
answers
47
views
hidden state of each sequence of mini-batch
I am new to Pytorch and trying to implement a lstm character level seq2seq model.
What I am trying to do is:
Each sequence is a list of the characters of a particular word and several words will ...
- 521
1
vote
0
answers
159
views
What is the correct formatting of the input tensor for multi-variate LSTM on Pytorch?
I am working on a LSTM to predict a financial time series using 10 other financial time series. The 10 financial time series form my training dataset.
Several examples I have seen for univariate ...
- 161
2
votes
0
answers
1k
views
LSTM not converging
I am sorry if this questions is basic but I am quite new to NN in general. I am trying to build an LSTM to predict certain properties of a light curve (the output is 0 or 1). I build it in pytorch. ...
- 21
1
vote
0
answers
313
views
LSTM Produces Random Predictions
I have trained an LSTM in PyTorch on financial data where a series of 14 values predicts the 15th. I split the data into Train, Test, and Validation sets. I trained the model until the loss ...
- 998
2
votes
2
answers
3k
views
Dropout Decreases Test and Train Accuracy in one layer LSTM in Pytorch
I have a one layer lstm with pytorch on Mnist data. I know that for one layer lstm dropout option for lstm in pytorch does not operate. So, I have added a drop out at the beginning of second layer ...
- 33