Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

Effective Time Series Forecasting using Keras/LSTM

I am working on time series forecasting for an engineering component (turbo charger). I have dataset containing field data from sensors (=features) taken every day for different turbocharger for their ...
0
votes
1answer
10 views

Intuition behind the RNN/LSTM hidden state?

What's the intuition behind the hidden states of RNN/LSTM? Are they similar to the hidden states of HMM (Hidden Markov Model)? Thanks!
1
vote
1answer
61 views

What model should I use for multiple time series input

I want to predict bacteria plate count in the water from time series(around 10000 values in a row) of water temperature on a one minute granularity, and other daily climate data including min and max ...
0
votes
0answers
10 views

is it possible to implement LSTM with input shape (sample,timestep,timestep,feature)?

I'm new to Keras. I am trying to implement this model https://www.aclweb.org/anthology/D15-1167 for document classification, and I want to use LSTM for getting sentence representation. I have trained ...
0
votes
0answers
6 views

Static and Dynamic neural networks process

Neural networks can be classified into static (convention feedforward networks) and dynamic categories (RNN, LSTMs). ...
1
vote
1answer
27 views

Structure of LSTM gates

It is my impression that a single layer LSTM architecture consists of $t$ LSTM cells that are identical duplicates, where $t$ is the number of time steps. Then there are gates within the LSTM cell. I ...
0
votes
0answers
10 views

Machine learning algorithm for classifying a 2xN array of ranged coordinates?

Good afternoon, I have a dataset of lists of coordinates that are ranged from (0, 100) on the Y-axis and (0, 300) on the x-axis, with double precision. I'm looking into classifier algorithms that ...
0
votes
0answers
8 views

Accelerometer and Gyroscope features

I am having accelerometer and gyroscope reading along x,y,z axis and want to get motion direction info at each time step. What all feature extraction would be best suited for this type of requirement. ...
0
votes
0answers
7 views

What techniques can I use to find position relationship in group of elements?

I have 14,000 tagged documents. These are custom forms that our employees create and fill out. I need to build a model that will be able to classify the types of each input field of the form in order ...
0
votes
1answer
45 views

ReLU for combating the problem of vanishing gradient in RNN?

For solving the problem of vanishing gradients in feedforward neural networks, ReLU activation function can be used. When we talk about solving the vanishing gradient problem in RNN, we use a more ...
2
votes
0answers
21 views

How can RNN handle variable sized inputs?

I came across this answer which is specific to Keras. But my question is at concept level. I am getting confused, How can RNN handle variable size inputs? here Let us suppose we want to do a ...
1
vote
1answer
275 views

What's the principal difference between ANN,RNN,DNN and CNN?

I'm newer to deep learning domain. I would like to know what is the principal difference between RNN,ANN,DNN and CNN? How to implement those neural networks using the TensorFlow library?
1
vote
1answer
177 views

2D-Input to LSTM in Keras

I have following problem: I would like to feed LSTM with train_datagen.flow_from_directory The input is basically a spectrogram images converted from time-series into time-frequency-domain in PNG ...
0
votes
0answers
5 views

To train in Keras two identical RNN but with different outputs

I am training to solve a problem where I am using a RNN to track an object, learn about it, and then generate trajectories. Therefore the input of the RNN is stuff like x,y,speed,... and the output is ...
0
votes
0answers
3 views

RNN batch outputs

Suppose, we have a set of variable length sequences. First: apply rnn unit to each sample and save outputs. Second: pad each sequence up to max_length with zeros at beginning. Apply same rnn unit to ...
0
votes
1answer
114 views

LSTM loss function and backpropagation

I'm trying to understand the connection between loss function and backpropagation. From what I understood until now, backpropagation is used to get and update matrices and bias used in forward ...
0
votes
1answer
34 views

How to convert sequence of words in to numbers which are input to RNN/LSTM?

I am watching online videos and tutorials about use of RNN/LSTM for NLP but none of them explain how to convert the sequences of words into digitized input to the neural networks? I am looking for ...
1
vote
0answers
21 views

Use pretrained model and create another sequential model

I have a pre-trained model, as below Code for the above model is as below,which is working as expected and I have saved this model separately in ".h5" keras format ...
0
votes
0answers
23 views

processing sequence of sequences in PyTorch

I try to deal with some special sequences via recurrent modules and I have faced some non-trivial things: Sequences are not in same length. I have sequence of sequences, and my idea is to encode ...
3
votes
0answers
36 views

Softmax activation predictions not summing to 1

I am a beginner with rnns, consider this sample code ...
0
votes
1answer
97 views

Training the document page layout and classifying good/bad layouts

I have a use case where I am supposed to get the coordinates of each block element in a page (whether its paragraph, image, table) where I train a model to understand how they are placed in a given ...
1
vote
1answer
69 views

LSTM number of units for first layer

I'm trying to use LSTM (with Keras) for a time series problem. I would like predict the next value of the time series given its previous value. I'm using TimeseriesGenerator to create the training ...
1
vote
0answers
20 views

Weird distribution of neural network outputs

I've faced an unusual behavior during training a neural network. The problem is to predict if a sample of 1st class or 2nd class. (2-class classification). Classes are imbalanced (~ 5 / 95). I use ...
0
votes
1answer
31 views

How to represent the number of neurons in an LSTM for architecture schematic?

I'm trying to visualise a neural network schematic and found a great tool for building schematics here http://alexlenail.me/NN-SVG/index.html. I've edited the SVG file to change one of the dense ...
0
votes
1answer
26 views

Contextual Spell Correction

I want to create a spell checker that corrects the spelling mistakes contextually. For example, Erroneous sentence: I want to apply for credit cart Corrected sentence: I want to apply for credit ...
0
votes
2answers
57 views

How to implement an LSTM RNN with multiple input features

EDIT: Now I didn't convert to list. I am training LSTM for multiple time-series in an array which has a structure: 450x801. There are 450 time series with each of 801 timesteps / time series. The ...
0
votes
0answers
24 views

Categorical Multivariate Time Series

I have a small dataset of products of which the price varies along time. Each product is represented by categorical features mostly ( type, matter, use, location ...) and one or two scalar features ( ...
0
votes
1answer
18 views

Very simple real-valued time-series dataset for RNN prototyping

Is there a simple real-valued time-series dataset on which a vanilla RNN model can be trained. With "very simple" I mean only two to four real-valued inputs per time step and a single real-valued ...
1
vote
0answers
12 views

How does the Backpropagation through time work?

I have to write a paper on LSTMs and I want to explain why LSTMs exist in the first place. According to some papers and books because usual RNNs had problems with vanishing gradients and the LSTM has ...
1
vote
0answers
51 views

Which one is better for handling spatio-temporal data: 3D CNN vs 2D Recurrent CNN?

Please forgive my ignorance and lack of experience: I am asking this question seeking answer from the experts/experienced persons in the field. I have a training dataset where each sample is a 3D ...
2
votes
1answer
67 views

LSTM number of units for time series

My question is on the number of units in an LSTM cell. I've come across the following example which is a model for predicting a value in a series based on its 2 lag observations. I'm wondering why ...
1
vote
2answers
22 views

BPTT vs Vanishing Gradient Problem

I know that BPTT is the method to apply Back Propagation on RNN. Which is works fine with RNN as it stops at certain point as changes approach to zero but isn't it the exact Vanishing Gradient ...
1
vote
0answers
18 views

Are there separate parameters/weights in LSTM cells for each timestep?

As I understand, the parameters and weights of a basic RNN is the same for each time step - there is only 1 set to train. Is this also the case for standard LSTM cell? More specifically, these are ...
0
votes
1answer
6 views

Confusion about Decoder labels for training seq-to-seq models

So in seq-to-seq models for say NMT, the decoder is a sequence model for the right-shifted intended output. My question is, during training, are the inputs and outputs of the decoder supposed to be ...
2
votes
1answer
18 views

Understanding the multidimensional-nature of the data being fed to a RNN and its output

Assuming we have a time-series dataset whose window_size = 30 and the batch_size = 4, which makes the overall input = 4*30 (2D). But as RNN expects 3D input, ...
0
votes
0answers
9 views

LSTM Captures Trend but Regresses to 0

I am using a vanilla LSTM to predict time series data. My simple model uses an 8 unit LSTM with dropout and a time distributed layer. The model can learn the shape of the data (i.e. when peaks and ...
0
votes
0answers
58 views

DA-RNN Keras implementation

Is there DA-RNN implementation with Keras or TensorFlow? If its a commented notebook it would be amazing https://arxiv.org/abs/1704.02971 here is the paper I am referring, I only found Torch ...
0
votes
0answers
33 views

LSTM model for many time series

Let's say one has many time series for which one wants to build a predictive model (based on LSTM). Which of the following cases would be more optimal and why? 1) building one model for all the time ...
0
votes
0answers
35 views

LSTM validation loss not improving

I'm a noob in the ML world and am currently building an LSTM to forecast the next page a user is going to visit on a website. My dataset is pretty much a mapping (with sliding window) from one page to ...
0
votes
0answers
28 views

Error when checking input: expected lstm_54_input to have shape (None, 5, 3) but got array with shape (1, 64, 3

I have been trying desperately to find an answer why my batch function (which returns the correct shape input sequence) is not compatible with my lstm architecture which throws an input shape error. ...
0
votes
0answers
21 views

Understanding about RNN loss

Problem I am now taking Andrew Ng's deep learning course on Coursera. Everything is great but when it comes to RNN, I sometimes feel confused. Here is a question about RNN (or more specifically, the ...
0
votes
0answers
40 views

How can we compute likelihood in recurrent neural networks?

Suppose that we have a recurrent neural network (RNN) with length $T$ for a classification task that generates an output at every time step which is a probability distribution over classes obtained by ...
0
votes
0answers
18 views

How to train a language model with bi lstm layers?

I am trying to understand how to train a LM using bi-LSTM in the case with "stack of bi LSTM". In the case of forward LSTM, we just need to add a classification layer on the top of the last hidden ...
0
votes
0answers
57 views

Keras error in dimensions when predicting

I have trained a Keras LSTM model and was now trying to use it for predictions but for some reason in is giving a dimensions error I cannot find. I processed the data in the same way as the training ...
2
votes
1answer
24 views

What Models should i try for this problem?

I need some advice for a problem i'm working on with automobile data. The vehicles provide a series of codes at every second which are bieng stored, though it can vary how many. For example , at time ...
0
votes
0answers
16 views

How LSTM compare which information is important or not?

I am interested to know, if I have scaled my data between [0,1], and have a vector like [0, 0.001, 0.01, 0.1, 1], is that mean ...
0
votes
1answer
34 views

LSTM get next output with Keras

So I'm learning RNN, and tried to do a prediction LSTM, but I do not understand how the output works. I have this LSTM RNN: ...
2
votes
1answer
573 views

in TensorFlow 2.0, what is the different between LSTM and LSTMCell objects?

I am trying to implement an RNN in TensorFlow 2.0 (beta1). Looking at the layer functions (inherited from Keras) I found: tf.keras.layers.LSTM and ...
0
votes
0answers
19 views

How to Reduce Overfitting of Deeplearning models on NLP tasks in unbalanced datasets?

I have a binary classification problem, where the number of examples belonging to Class 0 is 20% on average. And the rest 80% of examples fall into Class ...
0
votes
0answers
14 views

Difference between globalmaxpoolin1d() and attention layer

What's the difference between globalmaxpoolin1d() and attention layer?