Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Accelerometer and Gyroscope features

I am having accelerometer and gyroscope reading along x,y,z axis and want to get motion direction info at each time step. What all feature extraction would be best suited for this type of requirement. ...
0
votes
0answers
7 views

What techniques can I use to find position relationship in group of elements?

I have 14,000 tagged documents. These are custom forms that our employees create and fill out. I need to build a model that will be able to classify the types of each input field of the form in order ...
0
votes
1answer
16 views

ReLU for combating the problem of vanishing gradient in RNN?

For solving the problem of vanishing gradients in feedforward neural networks, ReLU activation function can be used. When we talk about solving the vanishing gradient problem in RNN, we use a more ...
1
vote
0answers
17 views

How can RNN handle variable sized inputs?

I came across this answer which is specific to Keras. But my question is at concept level. I am getting confused, How can RNN handle variable size inputs? here Let us suppose we want to do a ...
1
vote
1answer
96 views

What's the principal difference between ANN,RNN,DNN and CNN?

I'm newer to deep learning domain. I would like to know what is the principal difference between RNN,ANN,DNN and CNN? How to implement those neural networks using the TensorFlow library?
1
vote
1answer
80 views

2D-Input to LSTM in Keras

I have following problem: I would like to feed LSTM with train_datagen.flow_from_directory The input is basically a spectrogram images converted from time-series into time-frequency-domain in PNG ...
0
votes
0answers
5 views

To train in Keras two identical RNN but with different outputs

I am training to solve a problem where I am using a RNN to track an object, learn about it, and then generate trajectories. Therefore the input of the RNN is stuff like x,y,speed,... and the output is ...
0
votes
0answers
3 views

RNN batch outputs

Suppose, we have a set of variable length sequences. First: apply rnn unit to each sample and save outputs. Second: pad each sequence up to max_length with zeros at beginning. Apply same rnn unit to ...
0
votes
1answer
24 views

LSTM loss function and backpropagation

I'm trying to understand the connection between loss function and backpropagation. From what I understood until now, backpropagation is used to get and update matrices and bias used in forward ...
0
votes
1answer
30 views

How to convert sequence of words in to numbers which are input to RNN/LSTM?

I am watching online videos and tutorials about use of RNN/LSTM for NLP but none of them explain how to convert the sequences of words into digitized input to the neural networks? I am looking for ...
1
vote
0answers
21 views

Use pretrained model and create another sequential model

I have a pre-trained model, as below Code for the above model is as below,which is working as expected and I have saved this model separately in ".h5" keras format ...
0
votes
0answers
22 views

processing sequence of sequences in PyTorch

I try to deal with some special sequences via recurrent modules and I have faced some non-trivial things: Sequences are not in same length. I have sequence of sequences, and my idea is to encode ...
3
votes
0answers
31 views

Softmax activation predictions not summing to 1

I am a beginner with rnns, consider this sample code ...
0
votes
1answer
94 views

Training the document page layout and classifying good/bad layouts

I have a use case where I am supposed to get the coordinates of each block element in a page (whether its paragraph, image, table) where I train a model to understand how they are placed in a given ...
1
vote
1answer
59 views

LSTM number of units for first layer

I'm trying to use LSTM (with Keras) for a time series problem. I would like predict the next value of the time series given its previous value. I'm using TimeseriesGenerator to create the training ...
1
vote
0answers
19 views

Weird distribution of neural network outputs

I've faced an unusual behavior during training a neural network. The problem is to predict if a sample of 1st class or 2nd class. (2-class classification). Classes are imbalanced (~ 5 / 95). I use ...
0
votes
1answer
22 views

How to represent the number of neurons in an LSTM for architecture schematic?

I'm trying to visualise a neural network schematic and found a great tool for building schematics here http://alexlenail.me/NN-SVG/index.html. I've edited the SVG file to change one of the dense ...
0
votes
1answer
26 views

Contextual Spell Correction

I want to create a spell checker that corrects the spelling mistakes contextually. For example, Erroneous sentence: I want to apply for credit cart Corrected sentence: I want to apply for credit ...
0
votes
2answers
49 views

How to implement an LSTM RNN with multiple input features

EDIT: Now I didn't convert to list. I am training LSTM for multiple time-series in an array which has a structure: 450x801. There are 450 time series with each of 801 timesteps / time series. The ...
0
votes
0answers
21 views

Categorical Multivariate Time Series

I have a small dataset of products of which the price varies along time. Each product is represented by categorical features mostly ( type, matter, use, location ...) and one or two scalar features ( ...
0
votes
1answer
18 views

Very simple real-valued time-series dataset for RNN prototyping

Is there a simple real-valued time-series dataset on which a vanilla RNN model can be trained. With "very simple" I mean only two to four real-valued inputs per time step and a single real-valued ...
1
vote
0answers
11 views

How does the Backpropagation through time work?

I have to write a paper on LSTMs and I want to explain why LSTMs exist in the first place. According to some papers and books because usual RNNs had problems with vanishing gradients and the LSTM has ...
1
vote
0answers
49 views

Which one is better for handling spatio-temporal data: 3D CNN vs 2D Recurrent CNN?

Please forgive my ignorance and lack of experience: I am asking this question seeking answer from the experts/experienced persons in the field. I have a training dataset where each sample is a 3D ...
2
votes
1answer
58 views

LSTM number of units for time series

My question is on the number of units in an LSTM cell. I've come across the following example which is a model for predicting a value in a series based on its 2 lag observations. I'm wondering why ...
1
vote
2answers
20 views

BPTT vs Vanishing Gradient Problem

I know that BPTT is the method to apply Back Propagation on RNN. Which is works fine with RNN as it stops at certain point as changes approach to zero but isn't it the exact Vanishing Gradient ...
1
vote
0answers
18 views

Are there separate parameters/weights in LSTM cells for each timestep?

As I understand, the parameters and weights of a basic RNN is the same for each time step - there is only 1 set to train. Is this also the case for standard LSTM cell? More specifically, these are ...
0
votes
1answer
6 views

Confusion about Decoder labels for training seq-to-seq models

So in seq-to-seq models for say NMT, the decoder is a sequence model for the right-shifted intended output. My question is, during training, are the inputs and outputs of the decoder supposed to be ...
2
votes
1answer
18 views

Understanding the multidimensional-nature of the data being fed to a RNN and its output

Assuming we have a time-series dataset whose window_size = 30 and the batch_size = 4, which makes the overall input = 4*30 (2D). But as RNN expects 3D input, ...
0
votes
0answers
9 views

LSTM Captures Trend but Regresses to 0

I am using a vanilla LSTM to predict time series data. My simple model uses an 8 unit LSTM with dropout and a time distributed layer. The model can learn the shape of the data (i.e. when peaks and ...
0
votes
0answers
43 views

DA-RNN Keras implementation

Is there DA-RNN implementation with Keras or TensorFlow? If its a commented notebook it would be amazing https://arxiv.org/abs/1704.02971 here is the paper I am referring, I only found Torch ...
0
votes
0answers
32 views

LSTM model for many time series

Let's say one has many time series for which one wants to build a predictive model (based on LSTM). Which of the following cases would be more optimal and why? 1) building one model for all the time ...
0
votes
0answers
27 views

LSTM validation loss not improving

I'm a noob in the ML world and am currently building an LSTM to forecast the next page a user is going to visit on a website. My dataset is pretty much a mapping (with sliding window) from one page to ...
0
votes
0answers
27 views

Error when checking input: expected lstm_54_input to have shape (None, 5, 3) but got array with shape (1, 64, 3

I have been trying desperately to find an answer why my batch function (which returns the correct shape input sequence) is not compatible with my lstm architecture which throws an input shape error. ...
0
votes
0answers
18 views

Understanding about RNN loss

Problem I am now taking Andrew Ng's deep learning course on Coursera. Everything is great but when it comes to RNN, I sometimes feel confused. Here is a question about RNN (or more specifically, the ...
0
votes
0answers
33 views

How can we compute likelihood in recurrent neural networks?

Suppose that we have a recurrent neural network (RNN) with length $T$ for a classification task that generates an output at every time step which is a probability distribution over classes obtained by ...
0
votes
0answers
16 views

How to train a language model with bi lstm layers?

I am trying to understand how to train a LM using bi-LSTM in the case with "stack of bi LSTM". In the case of forward LSTM, we just need to add a classification layer on the top of the last hidden ...
0
votes
0answers
38 views

Keras error in dimensions when predicting

I have trained a Keras LSTM model and was now trying to use it for predictions but for some reason in is giving a dimensions error I cannot find. I processed the data in the same way as the training ...
2
votes
1answer
24 views

What Models should i try for this problem?

I need some advice for a problem i'm working on with automobile data. The vehicles provide a series of codes at every second which are bieng stored, though it can vary how many. For example , at time ...
0
votes
0answers
16 views

How LSTM compare which information is important or not?

I am interested to know, if I have scaled my data between [0,1], and have a vector like [0, 0.001, 0.01, 0.1, 1], is that mean ...
0
votes
1answer
26 views

LSTM get next output with Keras

So I'm learning RNN, and tried to do a prediction LSTM, but I do not understand how the output works. I have this LSTM RNN: ...
2
votes
1answer
363 views

in TensorFlow 2.0, what is the different between LSTM and LSTMCell objects?

I am trying to implement an RNN in TensorFlow 2.0 (beta1). Looking at the layer functions (inherited from Keras) I found: tf.keras.layers.LSTM and ...
0
votes
0answers
19 views

How to Reduce Overfitting of Deeplearning models on NLP tasks in unbalanced datasets?

I have a binary classification problem, where the number of examples belonging to Class 0 is 20% on average. And the rest 80% of examples fall into Class ...
0
votes
0answers
14 views

Difference between globalmaxpoolin1d() and attention layer

What's the difference between globalmaxpoolin1d() and attention layer?
0
votes
0answers
11 views

How to generate sequence of text(overall trend) by reading stock price

I would like to generate a sequence of text by reading stock price, this sequence text should contain describing the trend of the stock prices and trajectory. There are two types of input sources, ...
0
votes
0answers
39 views

In an RNN, if the gradients don't vanish for long/distant terms, won't the derivative of the error be either divergent to infinity or oscillatory?

P.S. Crosss posted here- https://stats.stackexchange.com/questions/413843/in-an-rnn-if-the-gradients-dont-vanish-for-long-distant-terms-wont-the-deriv, as I've got no answer, I'm asking here: In my ...
0
votes
2answers
75 views

How to predict next visit date based on this data

I have a dataset shown below. Here, status is if visit has been done or not and schedule is if next_action_scheduled. ...
0
votes
1answer
25 views

Understanding Layers in Recurrent Neural Networks for NLP

In convolution neural networks, we have a concept that inner layers learn fine features like lines and edges, while outer layers learn more complex shapes. Do we have any such understanding for ...
0
votes
0answers
25 views

Detect Typical Customer Mistakes in the Shopping Cart

I would like to ask your advice on solving this problem. Problem: There is an online store which sells furniture. There are millions of products on the store like furniture parts and furniture ...
0
votes
1answer
22 views

How to scale exponential data for a regression problem?

I understand that I should be scaling features between (0, 1) before feeding them into a neural network. However, what happens if future data could be larger than my current training data? For ...