Questions tagged [recurrent-neural-net]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Do Any Frameworks Provide Better Support for End-To-End Integer-Based Feature Engineering, Modeling, and Inference?

A retail enterprise I work with with wants to switch from its home-grown time series data analysis and prediction system to something more established and with community support. One unique feature ...
0
votes
0answers
13 views

How to put marker on time series training set

My input is this picture And I would like to put markers on it and use time series with markers as a label Two picture are not scaled. Main point is I would like to train RNN classifier and let it be ...
0
votes
0answers
12 views

Pytorch: how to pass the hidden state between the samples in LSTM?

I am trying to boost the performance of a object detection task with sequential information, using ConvLSTM. A typical ConvLSTM model takes a 5D tensor with shape ...
0
votes
0answers
6 views

What is the difference between the second layer of a recurrent hamming network and a simple maximum search?

In a Hamming network, the second Layer is a recurrent/competitive layer, in which its neurons are initialized by the outputs of first layer see chapter 3-10 in Neural Network Design. The output $a$ ...
0
votes
1answer
71 views

Problems to understand how to create the input data for time series forecasting with a recurrent neural network in Keras

I just started to use recurrent neural networks (RNN) with Keras for time-series forecasting and I found this tutorial Forecasting with RNN. I have difficulties understanding how to build the training ...
0
votes
0answers
60 views

Need help to understand backpropagation for gated recurrent units (GRU)

I'm stuck regarding the implementation of backpropagation in a binary classification task using a GRU. I wanted to know if someone could tell me how to proceed. I was able to understand how BP works ...
-1
votes
2answers
20 views

Optimisation of neural networks

Do neural networks get optimized by trial and error, by data scientists, or is there some way of optimizing values through accurate mathematical equations?
0
votes
0answers
29 views

Output of a network to it's input, keras

I'm trying to create a neural network in keras for time series forecast. I've build a concept, and now I'm not quite sure if it is possible to implement using keras. I have a potentially complicated ...
0
votes
1answer
23 views

Tricky stacking models in keras

I'm trying to write a model with keras, that is built as shown below: ...
0
votes
1answer
16 views

Difference between zero-padding and character-padding in Recurrent Neural Networks

For RNN's to work efficiently we vectorize the problem which results in an input matrix of shape (m, max_seq_len) where m is the number of examples, e.g. ...
1
vote
1answer
32 views

Timeseries LSTM: does test data need to come after training data?

I have one single, very long time series. I want to train an LSTM to distinguish between two behaviours (A or B) at every timestep (sequence-to-sequence). Because the time series is very long, I plan ...
0
votes
0answers
27 views

Making Keras RNN to proceed input sequence step by step

I'm currently trying to create a neural network for playing Tetris. I'm using evolutionary algorythms for it's learning, so the behavior that I need to get from the neural network is the following: ...
1
vote
1answer
27 views

Number of parameters in Simple RNNs

Please, I am stuck, I can not understand the number of parameters of a simple RNN, here the example and the model summary. the example is simple: ...
0
votes
0answers
12 views

RNN Layer with multiple independent sequences for same label

I have a question about the use of longitudinal data (reoccurring sequences of data). Let's imagine we have multiple independent longitudinal features (e.g. visits to doctor and purchases in online ...
1
vote
0answers
49 views

Concatenation of CNN and LSTM to model time of a series of images

I have collected a dataset consisting of around 30'000 heat maps of 80 users. The heat maps represent typing behavior on a keyboard and are just images with a resolution of ...
0
votes
0answers
30 views

How to account for rare events at different time intervals while using LSTM neural networks?

I'm working on an interesting sequence-to-sequence (regression) time series problem where some static features/rare events can change the behavior of future time series. The problem is a forecasting ...
2
votes
2answers
110 views

Understanding of number of cells in layers of sequential models

I am trying to teach myself RNN, but I have a question. And so, imagine 2 layers: an input layer with three neurons $(x1, x2, x3)$ and a classic recurrent layer with 2 neurons and an activation ...
2
votes
1answer
35 views

Modeling Encoder-Decoder according to instructions from a paper

I am new to this field and I was reading a paper "Predicting citation counts based on deep neural network learning techniques". There the authors describe the code that they implemented if ...
0
votes
0answers
10 views

Does Tensorflow allow recurrent connections across layers?

What i mean is: Allow any arbitrary layer to receive outputs from a layer downstream from a previous timestep as input. The network graph would be cyclic. I do not mean explicitly recurrent layers ...
0
votes
0answers
12 views

Can we do autoregressive using pad_packed_sequence?

I’m curious about if we want to do the autoregressive manner. Is it possible to do with implemented using pad_packed_sequence and pack_padded_sequence input to some recurrent network? Because we need ...
3
votes
0answers
53 views

How is the input gate in the LSTM learn?

How is the input gate neural network trained what to remember by propagating the error rate from predicting the next word in the language model? How does it help it to learn if it remembered the right ...
0
votes
0answers
93 views

What is the time complexity for training a gated recurrent unit (GRU) neural network using back-propagation through time?

Let us assume we have a GRU network containing $H$ layers to process a training dataset with $K$ tuples, $I$ features, and $H_i$ nodes in each layer. I have a pretty basic idea how the complexity of ...
1
vote
1answer
241 views

what is the complexity of a bidirectional recurrent neural network?

In particular, what is the complexity of a bi-directional recurrent neural network taking into account the variants of LSTM and GRU as well for training? I am hoping if I can get links to some ...
0
votes
0answers
21 views

How to get the weight matrices of intermediate layers in bidirectional recurrent neural networks?

I am a newbie in deep learning. I have a doubt regarding the training procedure of bidirectional recurrent neural networks using backpropagation through time. Following the original paper for ...
2
votes
1answer
339 views

Difference between Jordan, Elman and normal RNN

As far as I know for history, the Jordan network was proposed first in 1986 as a form of RNN with this diagram: Actually, this is the solution that makes sense when thinking about sequence data that ...
2
votes
2answers
57 views

What if we input sequence data to feedforward network?

One main advantage of RNN is the ability to take input of variable length like the case of sequences. However, what if we neglected this case and assumed some applications that may accept some fixed ...
1
vote
1answer
38 views

How does “one-to-many” RNNs work?

I recently came across an article about RNNs here. Which describes different types of RNNs like: The first figure makes sense. A regular feedforward network. The second is a big question for me. Is ...
0
votes
0answers
38 views

Google Trax's GRU layer

I am learning about Trax for the implementation of GRU and LSTMs. Their documentation says that a GRU layer in Trax can only accept a number of hidden units equal to the number of elements in the ...
0
votes
0answers
12 views

Time-series data pipeline for Kafka + Spark?

I'm working on a project related to predicting syscall sequences' malice. The data, which are syscall sequences that each belongs to a program, will be stored temporary to Kafka (which I have no idea ...
1
vote
1answer
50 views

Clarify recurrent neural networks

I'm in the beginning to learn and understand recurrent neural networks. As far as I can imagine, its multiple feed-forward neural networks with one neuron at each layer put next to each other, and ...
8
votes
3answers
162 views

feature importance after classification

I have time series data and more or less 200 features for each sample, I used a recurrent neural network for the binary classification task. After the classification I would like to know which ...
0
votes
0answers
23 views

Why is the accuracy on the test dataset very low when training a neural network on an IMU dataset?

I am trying to train an IMU (Inertial Measurement Unit) dataset. The dataset contain 6 features (3-gyro, 3-accelerometer) and 1 label column. I have build a neural network via Conv1D, LSTM and Dense ...
0
votes
0answers
14 views

What is the contraction map constraint in the context of Graph Neural Nets?

In this paper https://arxiv.org/pdf/1511.05493.pdf (GATED GRAPH SEQUENCE NEURAL NETWORKS,2016), it is stated that in a Graph Neural network initialising hidden states is not required, as 'In GNNs, ...
0
votes
2answers
123 views

What are the hidden states in the Transformer-XL? Also, how does the recurrence wiring look like?

After exhaustively reading the many blogs and papers on Transformers-XL, I still have some questions before I can say that I understand Transformer-XL (and by extension XLNet). Any help in this regard ...
0
votes
1answer
40 views

Does LSTM without delayed inputs work as a deep net?

I want to predict a multivariate time series. My time series is $a_1(t),...,a_k(t)$ and I want to predict $a_k(t)$. I use the following keras LSTM: ...
2
votes
1answer
27 views

GRU and LSTM does not “take risk” predicting

I tested LSTM and GRU models to predict the exchange rate between currencies. I do not take the raw price but a the delta with the previous day, so the data is stationnary around zero. My problem is ...
0
votes
1answer
34 views

Can someone explain to me the structure of a plain Recurrent Neural Network?

I have seen pictures of RNNs and LTSMs, and they usually look like this: Here the task is to take a sentence and make a prediction of some sort. What are each of the green squares? Are each of them ...
1
vote
1answer
28 views

What are some solutions for dealing with time series data that are recorded at uneven intervals?

Let's say I have a time series data which is a bunch of observations that occur at different time stamps and intervals. For example, my observations come from a camera located at a traffic ...
1
vote
0answers
17 views

What is External representation of time in Sequential learning?

I am reading the literature on sequential learning and it is often mention that in order to handle sequential/temporal data, there two categories of work in sequential learning External ...
1
vote
0answers
31 views

Train an RNN with image sequences of varying length in keras for regression

How to program a RNN model with image sequences of varying length in keras for regression? (Just assume the output is some predefined continuous values) I've read up things about training RNN on ...
0
votes
0answers
73 views

Why does the forecasting of this LSTM model look like a steady line?

This is a multivariate multistep problem using LSTM NN model. I am trying to forecast one variable by means of the other variables. However, the forecasting output looks like a horizontal line. Kindly ...
2
votes
0answers
45 views

LSTM low training/validation error but really bad predictions

I'm building a LSTM model to create an automatic drums composer. I'm following this post: LSTM Metallica I've built my model and done all the enconding, I was able to emulate the behavior of the ...
3
votes
0answers
16 views

How respective gating functions are ensured in LSTM?

I'm studying the Hochreiter-Schmidhuber long-short term memory recurrent architecture. The overall idea, information flow and manipulation is clear, and it seemingly works, but what I cannot ...
0
votes
0answers
7 views

How to include the other variables at t=t to predict the target variable with time lags also in LSTM?

I am having a training data set for a time-series dataset like below: ...
2
votes
1answer
68 views

fluctuating values for validation set only

My model's structure is ...
3
votes
3answers
2k views

Why are predictions from my LSTM Neural Network lagging behind true values?

I am running an LSTM neural network in R using the keras package, in an attempt to do time series prediction of Bitcoin. The issue I'm running into is that while my predicted values seem to be ...
1
vote
1answer
30 views

Loaded model predicts well in colab but gives same label and accuracy when downloaded

I have developed a Recurrent Neural Network to perform sentiment analysis on tweets using the Kazanova/sentiment140 dataset in Kaggle. The model looks like this: ...
2
votes
1answer
229 views

Can bidirectional RNN use variable sequence length?

A bidirectional RNN consists of two RNNs, one for the forward and another for the backward sequential directions, which outcome is concatenated at each time step. Would this configuration restrict the ...
0
votes
0answers
59 views

How to properly interpret the train and val loss?

I am currently doing some research in neural networks for regression problems. Following some plots of the train and validation loss of different models. The blue line is the train loss and the orange ...
1
vote
0answers
102 views

Implementing Dropout for Recurrent Layers in Keras + Theano

I am looking to implement recurrent dropout (where recurrent connections between memory units of a recurrent layer such as LSTM/GRU/RNN are randomly set to 0) in Keras 2.3.1 on Theano backend on ...