Questions tagged [stacked-lstm]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
52 views

What is the reason behind Keras choice of default (recurrent) activation functions in LSTM networks

Activation function between LSTM layers In the above link, the answer to the question whether activation function are required for LSTM layers was answered as follows: as an LSTM unit already consists ...
0
votes
0answers
47 views

Why does the forecasting of this LSTM model look like a steady line?

This is a multivariate multistep problem using LSTM NN model. I am trying to forecast one variable by means of the other variables. However, the forecasting output looks like a horizontal line. Kindly ...
0
votes
0answers
31 views

LSTM backpropagation issue in Keras code

I have trained the network with data in batches of some batch size >1. After training, I am using trained network and then manually update parameters for every example using backpropagation. The ...
0
votes
0answers
18 views

LSTM's with variable size input features + how to do embeddings with (x,y) coordinate systems

Background : So I have a dataset of x,y positions of dancers dancing("doin' their thang!!"). Some sequences of the dance are with 8, some with 5,4,8, upto 16. So, I am trying to do something like ...
10
votes
2answers
6k views

Activation function between LSTM layers

I'm aware the LSTM cell uses both sigmoid and tanh activation functions internally, however when creating a stacked LSTM architecture does it make sense to pass their outputs through an activation ...
2
votes
0answers
25 views

Wiggle in the initial part of an LSTM prediction

I working on using LSTMs and GRUs to make time series predictions. For the most part the predictions are pretty good. However, there seems to be a wiggle (or initial up-then-down) before the ...
2
votes
0answers
147 views

LSTM for multiple time series regression with extremely large ranges

I have the following question for those which encountered the same dilemma as me: My target is to develop a LSTM RNN for multi-step prediction for multiple time series representing daily sales of ...
0
votes
0answers
67 views

How multi layer LSTM are interconnected?

I am trying to understand the layers in LSTM for my own implementation using Python. I started with Keras to getting familiarized with the layer flow. I have tried the below code in Keras and I have ...
0
votes
1answer
223 views

What are h(t-1) and c(t-1) for the first LSTM cell?

I know in a LSTM chain you should connect the h(t) of the previous cell to the h(t+1) of the next cell, and doing so for c(t). But what about the first cell? What does it get as h(t-1) and c(t-1)? I ...
1
vote
0answers
242 views

Splitting and training multiple datasets at the same time

I've got 15 different datasets at about 10GB each. Each dataset comes with a binary 2D ground truth (10486147ish, 1) that I pull from it. I'm trying to figure out how to load each dataset, split them ...
1
vote
0answers
36 views

Initialising states in a multilayer sequence to sequence model

With a sequence to sequence model where the enocoder and decoder are both comprised of one layer each, the initial state of the decoder is initialised to use the final states of the encoder layer. In ...
3
votes
1answer
1k views

Connect a dense layer to a LSTM architecture

I am trying to implement an LSTM structure in plain numpy for didactic reason. I clearly understand how to input the data, but not how to output. Suppose I give as inputs a tensor of dimension (n, b, ...
1
vote
0answers
122 views

LSTM Produces Random Predictions

I have trained an LSTM in PyTorch on financial data where a series of 14 values predicts the 15th. I split the data into Train, Test, and Validation sets. I trained the model until the loss ...
0
votes
1answer
382 views

How can I detect anomalies/outliers in my online streaming data on a real-time basis?

Say, I've a huge set of data(infinite in size) consisting of alternating sine wave and step pulses one after the other. What I want from my model is to parse the data sequence wise or point wise and ...
2
votes
1answer
5k views

Stacking LSTM layers

Can someone please tell me the difference between those stacked LSTM layers? First image is given in this question and second image is given in this article. So far what I learned about stacking LSTM ...
1
vote
1answer
646 views

Why does my LSTM perform better when randomizing training subset vs. standard batch training?

I am training a simple LSTM network using Keras to predict time series values. It is a simple 2-layer LSTM. I get the best performance when I train on subsets of the training set that start at random ...
11
votes
2answers
12k views

Dropout on which layers of LSTM?

Using a multi-layer LSTM with dropout, is it advisable to put dropout on all hidden layers as well as the output Dense layers? In Hinton's paper (which proposed ...
3
votes
1answer
267 views

How is error back-propagated in a multi-layer RNN

Let's say I have a 2 layer LSTM cell, and I'm using this network to perform regression for input sequences of length 10 along the time axis. From what I understand, when this network is 'unfolded', ...