Questions tagged [recurrent-neural-net]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

How to include the other variables at t=t to predict the target variable with time lags also in LSTM?

I am having a training data set for a time-series dataset like below: ...
-1
votes
0answers
17 views

LSTM (Long Short-Term Memory) network usecase

I am new to LSTM and trying to put in a real life implementation, but not sure whether LSTM suits well. The use case is as following: There are many warehouses in different regions. Hourly, trucks ...
1
vote
1answer
32 views

fluctuating values for validation set only

My model's structure is ...
3
votes
3answers
1k views

Why are predictions from my LSTM Neural Network lagging behind true values?

I am running an LSTM neural network in R using the keras package, in an attempt to do time series prediction of Bitcoin. The issue I'm running into is that while my predicted values seem to be ...
1
vote
1answer
24 views

Loaded model predicts well in colab but gives same label and accuracy when downloaded

I have developed a Recurrent Neural Network to perform sentiment analysis on tweets using the Kazanova/sentiment140 dataset in Kaggle. The model looks like this: ...
2
votes
1answer
29 views

Can bidirectional RNN use variable sequence length?

A bidirectional RNN consists of two RNNs, one for the forward and another for the backward sequential directions, which outcome is concatenated at each time step. Would this configuration restrict the ...
0
votes
0answers
26 views

How to properly interpret the train and val loss?

I am currently doing some research in neural networks for regression problems. Following some plots of the train and validation loss of different models. The blue line is the train loss and the orange ...
1
vote
0answers
20 views

Implementing Dropout for Recurrent Layers in Keras + Theano

I am looking to implement recurrent dropout (where recurrent connections between memory units of a recurrent layer such as LSTM/GRU/RNN are randomly set to 0) in Keras 2.3.1 on Theano backend on ...
0
votes
1answer
16 views

Best way to handle padding in time series data such as text

I have a bunch of documents containing sequential data that I want to use to train a neural network with. It is as a collection of letters each about a 2-3000 characters long. My task is, given an ...
0
votes
1answer
34 views

How to understand Inconsistent and ambiguous dimensions of matrices used in the Attention layer?

Attention-scoring mechanism seems to be a commonly-used component in various seq2seq models, and I was reading about the original "Location-based Attention" in Bahadanau well-known paper at https://...
0
votes
0answers
22 views

Should LSTM data be a sequence?

let me explain what I want to do, I want to predict the trend of the price of something (1 if it increases in the next hour and 0 otherwise). I have gathered tweets about that and grouped them in ...
1
vote
0answers
13 views

how do deep Q network deal with varying input size?

I am conducting research with multiply agents in an environment. The main concept of my methodology is a centralized control system, which means we take the positions, as well as other information, of ...
0
votes
0answers
11 views

Is the number of bidirectional LSTMs in encoder-decoder model equal to the maximum length of input text/characters?

I'm confused about this aspect of RNNs while trying to learn how seq2seq encoder-decoder works at https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/. It ...
0
votes
0answers
8 views

Question about a basic aspect of how text-2-speech spectrogram frames are aligned?

A key aspect of how text-to-speech (TTS) machine-learning works is very unclear to me even after reading the Tacotron-2 paper and the Google AI blog. https://ai.googleblog.com/2017/12/tacotron-2-...
0
votes
1answer
24 views

Is vanilla RNN suitable for time series prediction?

I read this document: https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/ It was pretty simple, but I don't understand how to use it for predict the next sequence (for example) in trading ...
1
vote
1answer
31 views

Comparing Language Model of two corpora

I know using Conditional Language Model I can learn the probability of a sentence given the corpus I used to train my model. I will then be able to generate meaningful text by sampling from the ...
0
votes
0answers
15 views

Attention network without hidden state?

I was wondering how useful the encoder's hidden state is for an attention network. When I looked into the structure of an attention model, this is what I found a model generally looks like: ...
0
votes
0answers
9 views

Are neural networks modular? An example

BACKGROUND Consider a supervised problem which is based on two scalar features (1) and (2) as well as a third, "time-dependent", feature consisting of a sequence of five values (3)-(7). For ...
0
votes
0answers
15 views

Should I use Stateful or Stateless LSTM

I am trying to use LSTM in Keras and I am not sure whether I should used statefull or stateless LSTM. I have read many resources online but seem like they do not apply to my case. I have a long ...
1
vote
0answers
17 views

LSTM / GRU prediction with hidden state?

I am trying to predict a value based on time series by series of 24 periods (the 25th period) While training I have a validation set with I babysit the training (RMSE) and each epoch, eval the ...
1
vote
0answers
18 views

Whats the difference between add.LSTM(num_hidden, droput=0.5) and add.Dropout(0.5) in Keras?

Could anyone please explain what is the difference between these two cases, specified in the title. I believe I am not the only one who is confused. I have read that it is preferrable to add Dropout ...
0
votes
0answers
7 views

seq2seq - Inference model and train model produce far too different results on the same validation set

I am working on a timeseries seq2seq problem. For my approach, I am using LSTM seq2seq RNN's with Teacher Forcing. As you already know, for the purpose of the task a model should be trained, and then ...
1
vote
0answers
19 views

Multi-feature padding for LSTM

I am trying to train a LSTM on an NER dataset which contains multiple features. But I'm having trouble understanding how to pad multiple features. The dataset contains the following 3 features per row:...
0
votes
1answer
37 views

How do I use matrix math in irregular neural networks such as those generated from neuroevolution (NEAT)?

I understand how to structure the matrix when every node in a layer is fully connected to every node in adjacent layers and I understand that in "irregular" neural networks I can just process each ...
0
votes
0answers
13 views

Multiple Outputs LSTM

I am trying to create a neural network capable of classifying the type of music that a user normally listens to.The idea is that the neural network will receive a 2D input matrix. These matrix ...
1
vote
0answers
17 views

What's wrong with my backpropagation through time (BTT) calculation or how to multiple a scaled vector and a matrix without matching dimensions?

I am trying to make a pretty simple RNN from scracth, using only Numpy library of Python. At this moment I am having troubles with BTT as I do not know how to proceed with situation when a ...
0
votes
0answers
15 views

Is Difference Transformation required for LSTM?

I am Working on uni variate multiple step LSTM sequence prediction .My LSTM model is failing to give a good prediction on My Data.From some online blogs I saw that Difference Transform may reduce Data ...
0
votes
0answers
21 views

Backpropagation through time details clarification

The way I understand back-propagation in time could be implemented in the following way: Go through the provided sequence, store the resulting hidden states of the network Iterate through the ...
7
votes
1answer
217 views

A the end of a big DS project, should I make trained models available on GitHub?

I almost completed two big Data Science personal projects based on Deep Learning. They are the fanciest models I've implemented up to now, and I'm pushing all my code on GitHub. Do you advice to ...
0
votes
0answers
12 views

Long sequence prediction with model trained on short sequence

I'll start with a specific example. I would like to train model which predict vector of [0-1]. Values close to 1 on specific range indicates that in those timesteps is particular activation word (...
0
votes
0answers
45 views

How to implement a LSTM for multilabel classification problem?

I would like to develop an LSTM because I have a variable input matrix. I am zero-padding to a specific length of 800. However, I am not sure of how to classify a certain situation when each input ...
1
vote
1answer
62 views

How to code a simple forward propagation of recurrent neural networks?

I know the theory behind recurrent neural networks or RNN but I am confused about its implementation. This is an rnn equation I got from the web, I tried to code the forward propagation alone in ...
0
votes
0answers
20 views

Summarize events per ID

Data: Each corresponds to an event (a person's visit to the hospital, as an example). I have a series of data associated with this event (duration of visit, motive, etc...). Objective: Summarize the ...
0
votes
0answers
28 views

Increase dimension of RNN LSTM cell in Keras

I want to increase amount of recurrent weights in rnn or lstm cell. The idea is that RNN neuron takes prevois output as input. I want to increase amount of previous values taken as input. If you ...
1
vote
2answers
25 views

Which input to use when generating a new sequence

I want to use sequence-to-sequence architecture to generate sequences. My data has such structure ...
4
votes
2answers
170 views

RNN in pseudo-code

A few years ago, I understood the classical MLP neural network much better when I wrote an implementation from scratch (using only Python + Numpy, without using tensorflow). Now I'd like to do the ...
1
vote
0answers
15 views

Credit attribution for prediction in recurrent neural nets

Consider a recurrent neural net, which has access two inputs sequences x1,x2,x3,x4.... and s1,s2,s3,s4... It emits a predictions p1,p2,p3,p4.... where ...
2
votes
0answers
19 views

Basic questions about hamming network

I'm reading Hogan et al's Neural Network Design book very closely. I have run into a couple of questions about its presentation on Hamming networks. In particular, it says: The next network we ...
0
votes
0answers
31 views

Generalised Estimating Equation (GEE) vs. Recurrent Neural Network (RNN)

Has anyone looked into or know what is the difference between a GEE model and an RNN model in terms of what these two models are doing? Apart from the differences in structure of these two models ...
0
votes
0answers
11 views

Multimodal vs recurrent network for video-based FER

I am wondering what types of architectures are best to explore for doing video-based facial emotion recognition. Some application feature using an architecture spatial-temporal architecture of 2 CNNs, ...
1
vote
1answer
24 views

How to return states of LSTM in MXNet?

I used MXNet previously to beat keras+tensorflow accuracy in CNN regression models. Now I am trying to implement LSTM, which in keras runs fine: ...
0
votes
1answer
46 views

What is the output of multivariate LSTM model?

I am currently trying to build an LSTM model by using multivariate inputs, but I don't understand what exact output I am predicting. I am currently using 5 features in the data i.e. 'Time', 'Avg CPU ...
2
votes
1answer
33 views

Create an RNN on text sources with different lengths

I want to create an RNN to generate a new text based on many examples of existing texts of a certain format in the training data. The type of texts in the training data consists of 3 segments, like so:...
1
vote
1answer
666 views

What is a 'hidden state' in BERT output?

I'm trying to understand the workings and output of BERT, and I'm wondering how/why each layer of BERT has a 'hidden state'. I understand what RNN's have a 'hidden state' that gets passed to each ...
2
votes
2answers
96 views

What is the role of $W_{ax}, W_{aa}, W_{ay}$ in forward propagation in RNN? Are they hyperparameters? Why are they needed?

In RNN introduction in Coursera sequence model course, the following formula for forward propagation in RNN was introduced. What exactly is the role of $W_{ax}, W_{aa}, W_{ay}$? What do they do? In ...
1
vote
0answers
9 views

Scipy.optimize getting back x0 for optimization of input to recurrent neural network

I need help with this optimization problem which is either not getting solved at all or is taking a copious amount of time. I am trying to find optimized input to an RNN (GRU) model of a process ...
2
votes
0answers
40 views

Masking seems not working for missing values problem in LSTM

I am trying to use LSTM to predict time series in keras. My input data shape is (1000,6,1)(samples,timesteps,features). There is some missing data in different timesteps. For example,[2,1,1]=NaN,[3,4,...
1
vote
1answer
30 views

How can I compare my regressors?

I am trying to build a regressor for a dataset which gives info about students' school performance and the probability of getting admitted in the University of their choice. The first 5 observations ...
1
vote
1answer
24 views

How does backpropagation work with averaging layers?

I'm studying Word2Vec algorithm, and so far i understood that, in the case of input context bigger than 1 (so multiple words) we have our hidden layer that performs averaging between the inputs (as ...
1
vote
1answer
486 views

A cross-entropy loss explanation in simple words

Suppose I build a FNN model. The last layer is a classification layer with softmax activation. A cross-entropy loss is used to classify a problems, such as logistic regression. How would I calculate ...