Questions tagged [recurrent-neural-net]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

RNN Layer with multiple independent sequences for same label

I have a question about the use of longitudinal data (reoccurring sequences of data). Let's imagine we have multiple independent longitudinal features (e.g. visits to doctor and purchases in online ...
1
vote
0answers
21 views

Concatenation of CNN and LSTM to model time of a series of images

I have collected a dataset consisting of around 30'000 heat maps of 80 users. The heat maps represent typing behavior on a keyboard and are just images with a resolution of ...
0
votes
0answers
18 views

How to account for rare events at different time intervals while using LSTM neural networks?

I'm working on an interesting sequence-to-sequence (regression) time series problem where some static features/rare events can change the behavior of future time series. The problem is a forecasting ...
2
votes
2answers
102 views

Understanding of number of cells in layers of sequential models

I am trying to teach myself RNN, but I have a question. And so, imagine 2 layers: an input layer with three neurons $(x1, x2, x3)$ and a classic recurrent layer with 2 neurons and an activation ...
3
votes
1answer
34 views

Modeling Encoder-Decoder according to instructions from a paper

I am new to this field and I was reading a paper "Predicting citation counts based on deep neural network learning techniques". There the authors describe the code that they implemented if ...
0
votes
0answers
9 views

Does Tensorflow allow recurrent connections across layers?

What i mean is: Allow any arbitrary layer to receive outputs from a layer downstream from a previous timestep as input. The network graph would be cyclic. I do not mean explicitly recurrent layers ...
0
votes
0answers
9 views

Can we do autoregressive using pad_packed_sequence?

I’m curious about if we want to do the autoregressive manner. Is it possible to do with implemented using pad_packed_sequence and pack_padded_sequence input to some recurrent network? Because we need ...
3
votes
0answers
46 views

How is the input gate in the LSTM learn?

How is the input gate neural network trained what to remember by propagating the error rate from predicting the next word in the language model? How does it help it to learn if it remembered the right ...
0
votes
0answers
28 views

What is the time complexity for training a gated recurrent unit (GRU) neural network using back-propagation through time?

Let us assume we have a GRU network containing $H$ layers to process a training dataset with $K$ tuples, $I$ features, and $H_i$ nodes in each layer. I have a pretty basic idea how the complexity of ...
1
vote
1answer
69 views

what is the complexity of a bidirectional recurrent neural network?

In particular, what is the complexity of a bi-directional recurrent neural network taking into account the variants of LSTM and GRU as well for training? I am hoping if I can get links to some ...
0
votes
0answers
19 views

How to get the weight matrices of intermediate layers in bidirectional recurrent neural networks?

I am a newbie in deep learning. I have a doubt regarding the training procedure of bidirectional recurrent neural networks using backpropagation through time. Following the original paper for ...
2
votes
1answer
163 views

Difference between Jordan, Elman and normal RNN

As far as I know for history, the Jordan network was proposed first in 1986 as a form of RNN with this diagram: Actually, this is the solution that makes sense when thinking about sequence data that ...
2
votes
2answers
50 views

What if we input sequence data to feedforward network?

One main advantage of RNN is the ability to take input of variable length like the case of sequences. However, what if we neglected this case and assumed some applications that may accept some fixed ...
1
vote
1answer
35 views

How does “one-to-many” RNNs work?

I recently came across an article about RNNs here. Which describes different types of RNNs like: The first figure makes sense. A regular feedforward network. The second is a big question for me. Is ...
0
votes
0answers
22 views

Google Trax's GRU layer

I am learning about Trax for the implementation of GRU and LSTMs. Their documentation says that a GRU layer in Trax can only accept a number of hidden units equal to the number of elements in the ...
0
votes
0answers
12 views

Time-series data pipeline for Kafka + Spark?

I'm working on a project related to predicting syscall sequences' malice. The data, which are syscall sequences that each belongs to a program, will be stored temporary to Kafka (which I have no idea ...
1
vote
1answer
48 views

Clarify recurrent neural networks

I'm in the beginning to learn and understand recurrent neural networks. As far as I can imagine, its multiple feed-forward neural networks with one neuron at each layer put next to each other, and ...
8
votes
3answers
156 views

feature importance after classification

I have time series data and more or less 200 features for each sample, I used a recurrent neural network for the binary classification task. After the classification I would like to know which ...
0
votes
0answers
20 views

Why is the accuracy on the test dataset very low when training a neural network on an IMU dataset?

I am trying to train an IMU (Inertial Measurement Unit) dataset. The dataset contain 6 features (3-gyro, 3-accelerometer) and 1 label column. I have build a neural network via Conv1D, LSTM and Dense ...
0
votes
0answers
13 views

What is the contraction map constraint in the context of Graph Neural Nets?

In this paper https://arxiv.org/pdf/1511.05493.pdf (GATED GRAPH SEQUENCE NEURAL NETWORKS,2016), it is stated that in a Graph Neural network initialising hidden states is not required, as 'In GNNs, ...
0
votes
2answers
76 views

What are the hidden states in the Transformer-XL? Also, how does the recurrence wiring look like?

After exhaustively reading the many blogs and papers on Transformers-XL, I still have some questions before I can say that I understand Transformer-XL (and by extension XLNet). Any help in this regard ...
0
votes
1answer
32 views

Does LSTM without delayed inputs work as a deep net?

I want to predict a multivariate time series. My time series is $a_1(t),...,a_k(t)$ and I want to predict $a_k(t)$. I use the following keras LSTM: ...
1
vote
1answer
20 views

GRU and LSTM does not “take risk” predicting

I tested LSTM and GRU models to predict the exchange rate between currencies. I do not take the raw price but a the delta with the previous day, so the data is stationnary around zero. My problem is ...
0
votes
1answer
30 views

Can someone explain to me the structure of a plain Recurrent Neural Network?

I have seen pictures of RNNs and LTSMs, and they usually look like this: Here the task is to take a sentence and make a prediction of some sort. What are each of the green squares? Are each of them ...
1
vote
1answer
24 views

What are some solutions for dealing with time series data that are recorded at uneven intervals?

Let's say I have a time series data which is a bunch of observations that occur at different time stamps and intervals. For example, my observations come from a camera located at a traffic ...
1
vote
0answers
16 views

What is External representation of time in Sequential learning?

I am reading the literature on sequential learning and it is often mention that in order to handle sequential/temporal data, there two categories of work in sequential learning External ...
1
vote
0answers
20 views

Train an RNN with image sequences of varying length in keras for regression

How to program a RNN model with image sequences of varying length in keras for regression? (Just assume the output is some predefined continuous values) I've read up things about training RNN on ...
0
votes
0answers
47 views

Why does the forecasting of this LSTM model look like a steady line?

This is a multivariate multistep problem using LSTM NN model. I am trying to forecast one variable by means of the other variables. However, the forecasting output looks like a horizontal line. Kindly ...
2
votes
0answers
20 views

LSTM low training/validation error but really bad predictions

I'm building a LSTM model to create an automatic drums composer. I'm following this post: LSTM Metallica I've built my model and done all the enconding, I was able to emulate the behavior of the ...
3
votes
0answers
16 views

How respective gating functions are ensured in LSTM?

I'm studying the Hochreiter-Schmidhuber long-short term memory recurrent architecture. The overall idea, information flow and manipulation is clear, and it seemingly works, but what I cannot ...
0
votes
0answers
6 views

How to include the other variables at t=t to predict the target variable with time lags also in LSTM?

I am having a training data set for a time-series dataset like below: ...
2
votes
1answer
55 views

fluctuating values for validation set only

My model's structure is ...
3
votes
3answers
2k views

Why are predictions from my LSTM Neural Network lagging behind true values?

I am running an LSTM neural network in R using the keras package, in an attempt to do time series prediction of Bitcoin. The issue I'm running into is that while my predicted values seem to be ...
1
vote
1answer
28 views

Loaded model predicts well in colab but gives same label and accuracy when downloaded

I have developed a Recurrent Neural Network to perform sentiment analysis on tweets using the Kazanova/sentiment140 dataset in Kaggle. The model looks like this: ...
2
votes
1answer
126 views

Can bidirectional RNN use variable sequence length?

A bidirectional RNN consists of two RNNs, one for the forward and another for the backward sequential directions, which outcome is concatenated at each time step. Would this configuration restrict the ...
0
votes
0answers
48 views

How to properly interpret the train and val loss?

I am currently doing some research in neural networks for regression problems. Following some plots of the train and validation loss of different models. The blue line is the train loss and the orange ...
1
vote
0answers
64 views

Implementing Dropout for Recurrent Layers in Keras + Theano

I am looking to implement recurrent dropout (where recurrent connections between memory units of a recurrent layer such as LSTM/GRU/RNN are randomly set to 0) in Keras 2.3.1 on Theano backend on ...
1
vote
1answer
83 views

Best way to handle padding in time series data such as text

I have a bunch of documents containing sequential data that I want to use to train a neural network with. It is as a collection of letters each about a 2-3000 characters long. My task is, given an ...
0
votes
1answer
53 views

How to understand Inconsistent and ambiguous dimensions of matrices used in the Attention layer?

Attention-scoring mechanism seems to be a commonly-used component in various seq2seq models, and I was reading about the original "Location-based Attention" in Bahadanau well-known paper at https://...
0
votes
0answers
22 views

Should LSTM data be a sequence?

let me explain what I want to do, I want to predict the trend of the price of something (1 if it increases in the next hour and 0 otherwise). I have gathered tweets about that and grouped them in ...
1
vote
0answers
30 views

how do deep Q network deal with varying input size?

I am conducting research with multiply agents in an environment. The main concept of my methodology is a centralized control system, which means we take the positions, as well as other information, of ...
0
votes
1answer
25 views

Is the number of bidirectional LSTMs in encoder-decoder model equal to the maximum length of input text/characters?

I'm confused about this aspect of RNNs while trying to learn how seq2seq encoder-decoder works at https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/. It ...
0
votes
0answers
10 views

Question about a basic aspect of how text-2-speech spectrogram frames are aligned?

A key aspect of how text-to-speech (TTS) machine-learning works is very unclear to me even after reading the Tacotron-2 paper and the Google AI blog. https://ai.googleblog.com/2017/12/tacotron-2-...
0
votes
1answer
60 views

Is vanilla RNN suitable for time series prediction?

I read this document: https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/ It was pretty simple, but I don't understand how to use it for predict the next sequence (for example) in trading ...
1
vote
1answer
35 views

Comparing Language Model of two corpora

I know using Conditional Language Model I can learn the probability of a sentence given the corpus I used to train my model. I will then be able to generate meaningful text by sampling from the ...
1
vote
1answer
29 views

Attention network without hidden state?

I was wondering how useful the encoder's hidden state is for an attention network. When I looked into the structure of an attention model, this is what I found a model generally looks like: ...
0
votes
0answers
11 views

Are neural networks modular? An example

BACKGROUND Consider a supervised problem which is based on two scalar features (1) and (2) as well as a third, "time-dependent", feature consisting of a sequence of five values (3)-(7). For ...
0
votes
0answers
26 views

Should I use Stateful or Stateless LSTM

I am trying to use LSTM in Keras and I am not sure whether I should used statefull or stateless LSTM. I have read many resources online but seem like they do not apply to my case. I have a long ...
1
vote
0answers
25 views

LSTM / GRU prediction with hidden state?

I am trying to predict a value based on time series by series of 24 periods (the 25th period) While training I have a validation set with I babysit the training (RMSE) and each epoch, eval the ...
0
votes
0answers
29 views

Whats the difference between add.LSTM(num_hidden, droput=0.5) and add.Dropout(0.5) in Keras?

Could anyone please explain what is the difference between these two cases, specified in the title. I believe I am not the only one who is confused. I have read that it is preferrable to add Dropout ...