Questions tagged [recurrent-neural-net]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
10 views

Clarify recurrent neural networks

I'm in the beginning to learn and understand recurrent neural networks. As far as I can imagine, its multiple feed-forward neural networks with one neuron at each layer put next to each other, and ...
6
votes
3answers
111 views

feature importance after classification

I have time series data and more or less 200 features for each sample, I used a recurrent neural network for the binary classification task. After the classification I would like to know which ...
0
votes
0answers
16 views

Why is the accuracy on the test dataset very low when training a neural network on an IMU dataset?

I am trying to train an IMU (Inertial Measurement Unit) dataset. The dataset contain 6 features (3-gyro, 3-accelerometer) and 1 label column. I have build a neural network via Conv1D, LSTM and Dense ...
0
votes
0answers
10 views

What is the contraction map constraint in the context of Graph Neural Nets?

In this paper https://arxiv.org/pdf/1511.05493.pdf (GATED GRAPH SEQUENCE NEURAL NETWORKS,2016), it is stated that in a Graph Neural network initialising hidden states is not required, as 'In GNNs, ...
0
votes
2answers
34 views

What are the hidden states in the Transformer-XL? Also, how does the recurrence wiring look like?

After exhaustively reading the many blogs and papers on Transformers-XL, I still have some questions before I can say that I understand Transformer-XL (and by extension XLNet). Any help in this regard ...
0
votes
1answer
24 views

Does LSTM without delayed inputs work as a deep net?

I want to predict a multivariate time series. My time series is $a_1(t),...,a_k(t)$ and I want to predict $a_k(t)$. I use the following keras LSTM: ...
0
votes
1answer
18 views

GRU and LSTM does not “take risk” predicting

I tested LSTM and GRU models to predict the exchange rate between currencies. I do not take the raw price but a the delta with the previous day, so the data is stationnary around zero. My problem is ...
0
votes
1answer
25 views

Can someone explain to me the structure of a plain Recurrent Neural Network?

I have seen pictures of RNNs and LTSMs, and they usually look like this: Here the task is to take a sentence and make a prediction of some sort. What are each of the green squares? Are each of them ...
1
vote
1answer
20 views

What are some solutions for dealing with time series data that are recorded at uneven intervals?

Let's say I have a time series data which is a bunch of observations that occur at different time stamps and intervals. For example, my observations come from a camera located at a traffic ...
1
vote
0answers
12 views

What is External representation of time in Sequential learning?

I am reading the literature on sequential learning and it is often mention that in order to handle sequential/temporal data, there two categories of work in sequential learning External ...
1
vote
0answers
16 views

Train an RNN with image sequences of varying length in keras for regression

How to program a RNN model with image sequences of varying length in keras for regression? (Just assume the output is some predefined continuous values) I've read up things about training RNN on ...
0
votes
0answers
25 views

Why does the forecasting of this LSTM model look like a steady line?

This is a multivariate multistep problem using LSTM NN model. I am trying to forecast one variable by means of the other variables. However, the forecasting output looks like a horizontal line. Kindly ...
2
votes
0answers
16 views

LSTM low training/validation error but really bad predictions

I'm building a LSTM model to create an automatic drums composer. I'm following this post: LSTM Metallica I've built my model and done all the enconding, I was able to emulate the behavior of the ...
3
votes
0answers
16 views

How respective gating functions are ensured in LSTM?

I'm studying the Hochreiter-Schmidhuber long-short term memory recurrent architecture. The overall idea, information flow and manipulation is clear, and it seemingly works, but what I cannot ...
0
votes
0answers
6 views

How to include the other variables at t=t to predict the target variable with time lags also in LSTM?

I am having a training data set for a time-series dataset like below: ...
2
votes
1answer
37 views

fluctuating values for validation set only

My model's structure is ...
3
votes
3answers
1k views

Why are predictions from my LSTM Neural Network lagging behind true values?

I am running an LSTM neural network in R using the keras package, in an attempt to do time series prediction of Bitcoin. The issue I'm running into is that while my predicted values seem to be ...
1
vote
1answer
25 views

Loaded model predicts well in colab but gives same label and accuracy when downloaded

I have developed a Recurrent Neural Network to perform sentiment analysis on tweets using the Kazanova/sentiment140 dataset in Kaggle. The model looks like this: ...
2
votes
1answer
38 views

Can bidirectional RNN use variable sequence length?

A bidirectional RNN consists of two RNNs, one for the forward and another for the backward sequential directions, which outcome is concatenated at each time step. Would this configuration restrict the ...
0
votes
0answers
27 views

How to properly interpret the train and val loss?

I am currently doing some research in neural networks for regression problems. Following some plots of the train and validation loss of different models. The blue line is the train loss and the orange ...
1
vote
0answers
31 views

Implementing Dropout for Recurrent Layers in Keras + Theano

I am looking to implement recurrent dropout (where recurrent connections between memory units of a recurrent layer such as LSTM/GRU/RNN are randomly set to 0) in Keras 2.3.1 on Theano backend on ...
0
votes
1answer
20 views

Best way to handle padding in time series data such as text

I have a bunch of documents containing sequential data that I want to use to train a neural network with. It is as a collection of letters each about a 2-3000 characters long. My task is, given an ...
0
votes
1answer
40 views

How to understand Inconsistent and ambiguous dimensions of matrices used in the Attention layer?

Attention-scoring mechanism seems to be a commonly-used component in various seq2seq models, and I was reading about the original "Location-based Attention" in Bahadanau well-known paper at https://...
0
votes
0answers
22 views

Should LSTM data be a sequence?

let me explain what I want to do, I want to predict the trend of the price of something (1 if it increases in the next hour and 0 otherwise). I have gathered tweets about that and grouped them in ...
1
vote
0answers
16 views

how do deep Q network deal with varying input size?

I am conducting research with multiply agents in an environment. The main concept of my methodology is a centralized control system, which means we take the positions, as well as other information, of ...
0
votes
0answers
13 views

Is the number of bidirectional LSTMs in encoder-decoder model equal to the maximum length of input text/characters?

I'm confused about this aspect of RNNs while trying to learn how seq2seq encoder-decoder works at https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/. It ...
0
votes
0answers
9 views

Question about a basic aspect of how text-2-speech spectrogram frames are aligned?

A key aspect of how text-to-speech (TTS) machine-learning works is very unclear to me even after reading the Tacotron-2 paper and the Google AI blog. https://ai.googleblog.com/2017/12/tacotron-2-...
0
votes
1answer
33 views

Is vanilla RNN suitable for time series prediction?

I read this document: https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/ It was pretty simple, but I don't understand how to use it for predict the next sequence (for example) in trading ...
1
vote
1answer
34 views

Comparing Language Model of two corpora

I know using Conditional Language Model I can learn the probability of a sentence given the corpus I used to train my model. I will then be able to generate meaningful text by sampling from the ...
0
votes
0answers
16 views

Attention network without hidden state?

I was wondering how useful the encoder's hidden state is for an attention network. When I looked into the structure of an attention model, this is what I found a model generally looks like: ...
0
votes
0answers
11 views

Are neural networks modular? An example

BACKGROUND Consider a supervised problem which is based on two scalar features (1) and (2) as well as a third, "time-dependent", feature consisting of a sequence of five values (3)-(7). For ...
0
votes
0answers
19 views

Should I use Stateful or Stateless LSTM

I am trying to use LSTM in Keras and I am not sure whether I should used statefull or stateless LSTM. I have read many resources online but seem like they do not apply to my case. I have a long ...
1
vote
0answers
20 views

LSTM / GRU prediction with hidden state?

I am trying to predict a value based on time series by series of 24 periods (the 25th period) While training I have a validation set with I babysit the training (RMSE) and each epoch, eval the ...
1
vote
0answers
20 views

Whats the difference between add.LSTM(num_hidden, droput=0.5) and add.Dropout(0.5) in Keras?

Could anyone please explain what is the difference between these two cases, specified in the title. I believe I am not the only one who is confused. I have read that it is preferrable to add Dropout ...
0
votes
0answers
8 views

seq2seq - Inference model and train model produce far too different results on the same validation set

I am working on a timeseries seq2seq problem. For my approach, I am using LSTM seq2seq RNN's with Teacher Forcing. As you already know, for the purpose of the task a model should be trained, and then ...
1
vote
0answers
37 views

Multi-feature padding for LSTM

I am trying to train a LSTM on an NER dataset which contains multiple features. But I'm having trouble understanding how to pad multiple features. The dataset contains the following 3 features per row:...
0
votes
1answer
53 views

How do I use matrix math in irregular neural networks such as those generated from neuroevolution (NEAT)?

I understand how to structure the matrix when every node in a layer is fully connected to every node in adjacent layers and I understand that in "irregular" neural networks I can just process each ...
0
votes
0answers
22 views

Multiple Outputs LSTM

I am trying to create a neural network capable of classifying the type of music that a user normally listens to.The idea is that the neural network will receive a 2D input matrix. These matrix ...
1
vote
0answers
19 views

What's wrong with my backpropagation through time (BTT) calculation or how to multiple a scaled vector and a matrix without matching dimensions?

I am trying to make a pretty simple RNN from scracth, using only Numpy library of Python. At this moment I am having troubles with BTT as I do not know how to proceed with situation when a ...
0
votes
0answers
22 views

Is Difference Transformation required for LSTM?

I am Working on uni variate multiple step LSTM sequence prediction .My LSTM model is failing to give a good prediction on My Data.From some online blogs I saw that Difference Transform may reduce Data ...
0
votes
0answers
26 views

Backpropagation through time details clarification

The way I understand back-propagation in time could be implemented in the following way: Go through the provided sequence, store the resulting hidden states of the network Iterate through the ...
7
votes
1answer
220 views

A the end of a big DS project, should I make trained models available on GitHub?

I almost completed two big Data Science personal projects based on Deep Learning. They are the fanciest models I've implemented up to now, and I'm pushing all my code on GitHub. Do you advice to ...
0
votes
0answers
14 views

Long sequence prediction with model trained on short sequence

I'll start with a specific example. I would like to train model which predict vector of [0-1]. Values close to 1 on specific range indicates that in those timesteps is particular activation word (...
0
votes
0answers
56 views

How to implement a LSTM for multilabel classification problem?

I would like to develop an LSTM because I have a variable input matrix. I am zero-padding to a specific length of 800. However, I am not sure of how to classify a certain situation when each input ...
1
vote
1answer
87 views

How to code a simple forward propagation of recurrent neural networks?

I know the theory behind recurrent neural networks or RNN but I am confused about its implementation. This is an rnn equation I got from the web, I tried to code the forward propagation alone in ...
0
votes
0answers
20 views

Summarize events per ID

Data: Each corresponds to an event (a person's visit to the hospital, as an example). I have a series of data associated with this event (duration of visit, motive, etc...). Objective: Summarize the ...
0
votes
0answers
33 views

Increase dimension of RNN LSTM cell in Keras

I want to increase amount of recurrent weights in rnn or lstm cell. The idea is that RNN neuron takes prevois output as input. I want to increase amount of previous values taken as input. If you ...
1
vote
2answers
26 views

Which input to use when generating a new sequence

I want to use sequence-to-sequence architecture to generate sequences. My data has such structure ...
4
votes
2answers
232 views

RNN in pseudo-code

A few years ago, I understood the classical MLP neural network much better when I wrote an implementation from scratch (using only Python + Numpy, without using tensorflow). Now I'd like to do the ...
1
vote
0answers
15 views

Credit attribution for prediction in recurrent neural nets

Consider a recurrent neural net, which has access two inputs sequences x1,x2,x3,x4.... and s1,s2,s3,s4... It emits a predictions p1,p2,p3,p4.... where ...