Questions tagged [stacked-lstm]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
27 views

LSTM backpropagation issue in Keras code

I have trained the network with data in batches of some batch size >1. After training, I am using trained network and then manually update parameters for every example using backpropagation. The ...
0
votes
0answers
9 views

LSTM's with variable size input features + how to do embeddings with (x,y) coordinate systems

Background : So I have a dataset of x,y positions of dancers dancing("doin' their thang!!"). Some sequences of the dance are with 8, some with 5,4,8, upto 16. So, I am trying to do something like ...
6
votes
1answer
214 views

Activation function between LSTM layers

I'm aware the LSTM cell uses both sigmoid and tanh activation functions internally, however when creating a stacked LSTM architecture does it make sense to pass their outputs through an activation ...
2
votes
0answers
20 views

Wiggle in the initial part of an LSTM prediction

I working on using LSTMs and GRUs to make time series predictions. For the most part the predictions are pretty good. However, there seems to be a wiggle (or initial up-then-down) before the ...
1
vote
0answers
60 views

LSTM for multiple time series regression with extremely large ranges

I have the following question for those which encountered the same dilemma as me: My target is to develop a LSTM RNN for multi-step prediction for multiple time series representing daily sales of ...
0
votes
0answers
48 views

How multi layer LSTM are interconnected?

I am trying to understand the layers in LSTM for my own implementation using Python. I started with Keras to getting familiarized with the layer flow. I have tried the below code in Keras and I have ...
0
votes
0answers
25 views

Stacking model using 100 seeds

i want to build a stacking model either by PNN or SVM of different 3 classifiers SVM, KNN, PNN and useing their optimized best paramters retrieved using Randomized grid search method but i need help ...
0
votes
1answer
79 views

What are h(t-1) and c(t-1) for the first LSTM cell?

I know in a LSTM chain you should connect the h(t) of the previous cell to the h(t+1) of the next cell, and doing so for c(t). But what about the first cell? What does it get as h(t-1) and c(t-1)? I ...
1
vote
0answers
161 views

Splitting and training multiple datasets at the same time

I've got 15 different datasets at about 10GB each. Each dataset comes with a binary 2D ground truth (10486147ish, 1) that I pull from it. I'm trying to figure out how to load each dataset, split them ...
1
vote
0answers
28 views

Initialising states in a multilayer sequence to sequence model

With a sequence to sequence model where the enocoder and decoder are both comprised of one layer each, the initial state of the decoder is initialised to use the final states of the encoder layer. In ...
3
votes
1answer
692 views

Connect a dense layer to a LSTM architecture

I am trying to implement an LSTM structure in plain numpy for didactic reason. I clearly understand how to input the data, but not how to output. Suppose I give as inputs a tensor of dimension (n, b, ...
1
vote
0answers
78 views

LSTM Produces Random Predictions

I have trained an LSTM in PyTorch on financial data where a series of 14 values predicts the 15th. I split the data into Train, Test, and Validation sets. I trained the model until the loss ...
0
votes
1answer
247 views

How can I detect anomalies/outliers in my online streaming data on a real-time basis?

Say, I've a huge set of data(infinite in size) consisting of alternating sine wave and step pulses one after the other. What I want from my model is to parse the data sequence wise or point wise and ...
2
votes
1answer
3k views

Stacking LSTM layers

Can someone please tell me the difference between those stacked LSTM layers? First image is given in this question and second image is given in this article. So far what I learned about stacking LSTM ...
0
votes
1answer
357 views

Why does my LSTM perform better when randomizing training subset vs. standard batch training?

I am training a simple LSTM network using Keras to predict time series values. It is a simple 2-layer LSTM. I get the best performance when I train on subsets of the training set that start at random ...
11
votes
2answers
7k views

Dropout on which layers of LSTM?

Using a multi-layer LSTM with dropout, is it advisable to put dropout on all hidden layers as well as the output Dense layers? In Hinton's paper (which proposed ...
3
votes
1answer
139 views

How is error back-propagated in a multi-layer RNN

Let's say I have a 2 layer LSTM cell, and I'm using this network to perform regression for input sequences of length 10 along the time axis. From what I understand, when this network is 'unfolded', ...