Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
1
vote
1answer
402 views

How to code a simple forward propagation of recurrent neural networks?

I know the theory behind recurrent neural networks or RNN but I am confused about its implementation. This is an rnn equation I got from the web, I tried to code the forward propagation alone in ...
0
votes
0answers
29 views

Summarize events per ID

Data: Each corresponds to an event (a person's visit to the hospital, as an example). I have a series of data associated with this event (duration of visit, motive, etc...). Objective: Summarize the ...
0
votes
0answers
277 views

LSTM data preparation with multiple independent observation runs

I'm struggling to wrap my head around how to correctly deal with multiple independent time series observations for training an LSTM. If I simply concatenate my observation runs, then there will be an ...
1
vote
2answers
31 views

Which input to use when generating a new sequence

I want to use sequence-to-sequence architecture to generate sequences. My data has such structure ...
4
votes
2answers
777 views

RNN in pseudo-code

A few years ago, I understood the classical MLP neural network much better when I wrote an implementation from scratch (using only Python + Numpy, without using tensorflow). Now I'd like to do the ...
2
votes
2answers
1k views

How do Bahdanau - Luong Attentions use Query, Value, Key vectors?

In the latest TensorFlow 2.1, the tensorflow.keras.layers submodule contains AdditiveAttention() and ...
1
vote
0answers
16 views

Credit attribution for prediction in recurrent neural nets

Consider a recurrent neural net, which has access two inputs sequences x1,x2,x3,x4.... and s1,s2,s3,s4... It emits a predictions p1,p2,p3,p4.... where ...
1
vote
2answers
374 views

Language translation with convolutional neural network

Many examples of language translation neural networks: "the cat sat on the mat" -> [model] -> "le chat etait assis sur le tapis" use RNN, and in particular LSTM. See for example Sentences language ...
1
vote
0answers
45 views

Reinforcement learning in bidirectional RNN

I have been self-learning deep generative neural network for a while. I am okay with the basics but I really need some guidance and jump start. I have recently came across this paper “Bidirectional ...
3
votes
1answer
41 views

preprocessing time sequence

I have a long list of event (400 unique events, sequence ~10M long). I want to train an RNN to predict next event. The preprocessing steps i took are: (1) turning to OneHotEncoding using pandas: <...
2
votes
0answers
38 views

Basic questions about hamming network

I'm reading Hogan et al's Neural Network Design book very closely. I have run into a couple of questions about its presentation on Hamming networks. In particular, it says: The next network we ...
1
vote
1answer
59 views

Classification model using RNN(action detection)

1) Could it be useful to use RNN for classification problem?(e.g. to distinguish which action is taken: car is going, walking, digging, nothing). If 1 question is positive, how should RNN structure ...
1
vote
0answers
30 views

How can I do a sequence to sequence model (RNN / LSTM) with Keras with fixed length data?

What I'm trying to do seems so simple, but I can't find any examples online. First, I'm not working in language, so all of the embedding stuff adds needless ...
1
vote
1answer
53 views

How to return states of LSTM in MXNet?

I used MXNet previously to beat keras+tensorflow accuracy in CNN regression models. Now I am trying to implement LSTM, which in keras runs fine: ...
1
vote
0answers
28 views

Time Series Forecasting with RNN/LSTM/NARX

I have some experimental datasets (like 4 or 5), and each dataset has three time series data, say $u1(t)$, $u2(t)$, and $x(t)$. The three time series of each experiment are similar but not the same. ...
2
votes
1answer
19 views

Use LSTM to predict the proportion of steps with nonzero feature values

I am trying to do a simple regression for sequences. Each input $X_i$ is a $n=2000$ by 1 matrix, formatted as $n_i$ 0-s followed by $(n-n_i)$ 1-s. The output $y_i$ should be $n_i/n$, i.e. the ...
2
votes
1answer
112 views

What is the output of multivariate LSTM model?

I am currently trying to build an LSTM model by using multivariate inputs, but I don't understand what exact output I am predicting. I am currently using 5 features in the data as input data: ...
1
vote
0answers
37 views

Loss is decreasing correctly but upon prediction, totally wrong results

I've made a pretty basic stock prediction RNN with it's only input being the past stock price from apple. On this case, I want to input the apple stock price and the samsung stock price (2 features). ...
5
votes
2answers
7k views

Loss being outputed as nan in keras RNN

Since the first Epoch of the RNN, the loss value is being outputted as nan. Epoch 1/100 9787/9787 [==============================] - 22s 2ms/step - loss: nan I have normalized the data. ...
2
votes
1answer
59 views

Create an RNN on text sources with different lengths

I want to create an RNN to generate a new text based on many examples of existing texts of a certain format in the training data. The type of texts in the training data consists of 3 segments, like so:...
1
vote
1answer
117 views

Keras RNN (batch_size

I created an RNN model for text classification with the LSTM layer, but when I put the batch_size in the fit method, my model trained on the whole batch instead of just the mini-batch _size. This also ...
1
vote
2answers
61 views

h in LSTM increasing in size?

So I was reading about the LSTM architecture and I was having trouble understanding a certain aspect of it. This article mentions the step in question near the bottom of the page. Here is the image ...
2
votes
1answer
5k views

What is a 'hidden state' in BERT output?

I'm trying to understand the workings and output of BERT, and I'm wondering how/why each layer of BERT has a 'hidden state'. I understand what RNN's have a 'hidden state' that gets passed to each ...
2
votes
2answers
137 views

What is the role of $W_{ax}, W_{aa}, W_{ay}$ in forward propagation in RNN? Are they hyperparameters? Why are they needed?

In RNN introduction in Coursera sequence model course, the following formula for forward propagation in RNN was introduced. What exactly is the role of $W_{ax}, W_{aa}, W_{ay}$? What do they do? In ...
1
vote
0answers
11 views

Scipy.optimize getting back x0 for optimization of input to recurrent neural network

I need help with this optimization problem which is either not getting solved at all or is taking a copious amount of time. I am trying to find optimized input to an RNN (GRU) model of a process ...
4
votes
1answer
28 views

Metric to evaluate words generated by Neural Network

I have this task at hand and I would be grateful for some directions. Perfectly not the final solution as I would like to do it myself. Let's say I need to create new fruit names based on existing ...
2
votes
0answers
22 views

Training time-series regression RNN's

I'm looking for references on training time-series regression RNN models. For learning purposes I want to implement myself using autograd (or JAX) rather than a high level library. I cannot find ...
1
vote
1answer
918 views

how to apply feature selection on LSTM-RNN? [closed]

am doing my research using lstm-rnn algorithm. i have time-series and non time-series features. how to apply lstm on my dataset? and also how to apply feature selection mechanism to select features?
2
votes
0answers
377 views

Proper loss function for sequence prediction model with multi-step output

Consider a typical time series (sequence) prediction problem that use previous $k$ step historical features to predict the next step target. We use RNN model as an ...
2
votes
0answers
53 views

Is it possible to get prediction intervals in sequenced data RNN forecasting?

Is it possible to get prediction intervals in sequenced data RNN (keras+python)forecasting? For example: predicts your car sales or new purchase The question is: UserId 1 will change his car in the ...
0
votes
1answer
48 views

About batches in stateful RNN

..., to create proper consecutive batches, where the nth input sequence in a batch starts off exactly where the nth input sequence ended in the previous batch. Géron, Aurélien. Hands-On Machine ...
2
votes
0answers
205 views

Masking seems not working for missing values problem in LSTM

I am trying to use LSTM to predict time series in keras. My input data shape is (1000,6,1)(samples,timesteps,features). There is some missing data in different timesteps. For example,[2,1,1]=NaN,[3,4,...
6
votes
1answer
2k views

TensorFlow / Keras: What is stateful = True in LSTM layers?

Could you elaborate on this argument? I found the brief explanation from the docs unsatisfying: stateful: Boolean (default False). If True, the last state for each sample at index i in a batch will ...
8
votes
3answers
13k views

what is darknet and why is it needed for YOLO object detection?

what is darknet and why is it needed for YOLO object detection ? I read that its a neural network written in C , but why is it needed for YOLO object detection when we have lot of machine learning ...
2
votes
0answers
125 views

Setting up RNN in TensorFlow for time series forecast with variable input series lengths

I am building a model with keras for time series prediction. The structure of the problem is as follows: The input is a time series of 5 numeric features The ...
1
vote
1answer
37 views

About seq2seq networks

I have read that seq2seq is a network, similar to other networks types (CNN, RNN, ...). However, in my opinion it is actually an architecture for RNNs. Isn't that? For example, when input and output ...
1
vote
1answer
35 views

How can I compare my regressors?

I am trying to build a regressor for a dataset which gives info about students' school performance and the probability of getting admitted in the University of their choice. The first 5 observations ...
1
vote
1answer
190 views

Why cant RNN learn long term dependencies=?

In Colah's blog, he explain this. In theory, RNNs are absolutely capable of handling such “long-term dependencies.” A human could carefully pick parameters for them to solve toy problems of ...
1
vote
1answer
36 views

How does backpropagation work with averaging layers?

I'm studying Word2Vec algorithm, and so far i understood that, in the case of input context bigger than 1 (so multiple words) we have our hidden layer that performs averaging between the inputs (as ...
1
vote
0answers
1k views

A cross-entropy loss explanation in simple words [duplicate]

Suppose I build a FNN model. The last layer is a classification layer with softmax activation. A cross-entropy loss is used to classify a problems, such as logistic regression. How would I calculate ...
2
votes
1answer
250 views

should I shift a dataset to use it for Time series regression with RNN/LSTM?

I'm seeing this tutorial to know how to use LSTM to predict time series data and I noticed that he shifted the target/labels up so that the features are all in time t but the target is t+1 so my ...
1
vote
0answers
17 views

what are the step need to be followed inorder to retrain any pretrained neural network model?

what are the step need to be followed inorder to retrain any pretrained neural network model ? (How to load the pre-trained BERT model from local/colab directory? ) I tried to re train few pretrained ...
1
vote
1answer
26 views

Efficient recurrent network for sequences of varying length

Suppose I have a bunch of sequences of varying lenghts. The absolute majority of them are short, just a few dozens items long. However, very few of them are significantly longer - more than a hundred ...
0
votes
2answers
61 views

What Non-linearities are best in Denoising RNN Autoencoders and where should the go?

I’m employing a denoising RNN autoencoder for a project relating to motion capture data. This is my first time using auto encoder architectures and I was just wondering what non-linearities should be ...
2
votes
1answer
43 views

Why RNNs necessary for time series?

I got it that when using time series data, I have to use a RNN The highlight here is that the neurons receive their output again as an input, so they can take into account the previous step. But ...
2
votes
2answers
2k views

k-fold cross validation with RNNs

is it a good idea to use k-fold cross-validation in the recurrent neural network (RNN) to alleviate overfitting? A potential solution could be ...
0
votes
0answers
323 views

How do I disable libtorch warning

Recently I deployed a program using libtorch (PyTorch C++ API). The program run as expected but its gives me a warning. ...
4
votes
1answer
3k views

SymbolicException: Inputs to eager execution function cannot be Keras symbolic tensors

I am writing Encoder-Decoder architecture with Bahdanau Attention using tf.keras with TensorFlow 2.0. Below is my code This is working with TensorFlow 1.15 but getting the error in 2.0. you can check ...
2
votes
0answers
109 views

How to compare the complexity of different RNN cells?

I want to compare three different types of RNNs to decide which architecture can handle my data best. To do that, I want them to have the same complexity. Can I simply define the complexity by the ...
1
vote
1answer
62 views

In Deep Learning, how many kinds of Attention exist? And what is the history of Attention models? [closed]

How many definitions of attention are commonly employed for Deep Learning tasks? That's what I've encountered up to now: Self-attention Bahdanau Luong Multi-Head (used in Transformers) Could you ...

1
3 4
5
6 7
14