# Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

741 questions
Filter by
Sorted by
Tagged with
8 views

### Unorthodox Time Series Preprocessing methods

I am writing a school paper on preprocessing of time series. In the paper I want to test the unorthodox methods of time series preprocessing. I want to then feed the data to LSTM model and measure its ...
32 views

### Deriving the gradient hidden to hidden weights for backpropagation through time in a reccurent neural network

I'm currently working on deriving the the gradients of a simple recurrent neural networks weights with respect to the loss to update the weights through backpropagation. It's a super simple network, ...
• 23
13 views

### Can RNNs Learn Time-Bounded Distributions of Stochastic Processes and Time-Dependent Transition Densities?

I'm currently reading the book Applied Stochastic Differential Equations by Särkkä and Solin (link - page 234), where it is mentioned that one can calculate the maximum likelihood estimate (MLE) for a ...
1 vote
13 views

### Why does summing the losses for each step in an RNN not lead to convergence?

I have been training an RNN model to predict the next number in an arithmetic sequence (a, a+d, a+2d, a+3d, a+4d, ...). Initially, I summed the loss for each step ...
• 1,205
1 vote
58 views

### using a 2d matrix as a feature input to LSTM / RNN models

i am building an LSTM model to predict the combination of items that will be sold at a store level on a daily basis. Please note, this is an exploratory model and i have a good idea about the ...
21 views

### How to combine Embedding layer with 3D input and 2D input in Pytorch

This familiar with my ideas. How to use Embedding() with 3D tensor in Keras? I'm re-implementing some table-to-text papers using RNN-based seq2seq (like this one https://arxiv.org/pdf/1603.07771v3) ...
• 101
7 views

### How BPTT updating the weights while input are varing

how the RNN gets trained(BPTT) when the input size is varying because to update the weights the network has to be stable right please reply on this Thanks in advance
15 views

### Semantics Building In LSTM-Based Models - How does a LSTM is able to extract and represent long data using just one value (long-memory)

How does a LSTM is able to extract and represent long sequences with data while using just one value (long-memory / LM) to maintain all this information? If multiple value were used, it could be ...
67 views

### Fuzzy Name Matching with Machine Learning. Input data encoding

I have a huge dataset: Last name, first name, date of birth of Indian residents and I need to match them for similarity. The matching is fuzzy, the data looks like this (names are fictitious for the ...
19 views

### What is the shape of the hidden/cell state of convLSTM2D?

I am new to convLSTM2D and I understand how it works, however, I am confused about the shape of the hidden states at different epochs ...
• 111
29 views

### recognition of names, surnames and patronymics

is there an example of neural networks on Github or Kaggle that perform the task of recognizing identical surnames, first names and patronymics? I'm just learning neural networks so it's interesting ...
• 21
52 views

### Determining the threshold value for the neural network

I have a dataset with last name, first name, middle name of people participating in sporting events. I need to train a neural network that will match similar surnames, first names and patronymics. But ...
• 21
12 views

### Deep neural network is plateauing on a regression task

I'm training a deep neural network on temporal graph data. Currently, I'm trying to get a feel for how large / complex of a model I should aim for, so I'm trying to overfit to my smallest dataset. ...
1 vote
18 views

### Call volume prediction using LSTM and GRU

Machine Learning call volume prediction using LSTM and GRU I am trying to predict the number of incoming calls using LSTM and GRU I have done all the data preprocessing but upon training the model I ...
1 vote
25 views

### Are formulas in the article incorrect?

I am learning about backpropagation in LSTM. I have been studying an article and watching two videos on the topic. The videos 1 and 2 repeat all the information from the article, but with additional ...
• 43
18 views

### Drum sound classification using RNN issues - help needed

I am new to the field of machine learning, even tho I have solid background in semi-related fields (am control system engineer by trade) and as a hobby project I wanted to work a bit with sound ...
14 views

### Adding sliding window dimension to data causes error: "Expected 3D or 4D (batch mode) tensor ..."

I wrote a pytorch data loader which used to return data of shape (4,1,192,320) representing the 4 samples of single channel image, each of size ...
• 199
23 views

### Model Architecture for Time-Series Forecasting with Categorical and Multivariate Data

Context: I was looking at using an LSTM model to forecast the amount of gold gained for each of 10 heroes in a game of Dota 2, a MOBA game, as a base model in some type of model architecture. The game ...
• 1
45 views

### Connecting Flatten layer to Dense layer

I'm struggling with my neural network. In short, I need to recreate a model from anywhere on the internet, I've found a model that combines BiLSTM, LSTM and GRU. However, based on the error I got when ...
18 views

### Is it possible to calculate a GRU RNN in its entirety by hand on a small dataset?

I want to see whether my code works and compare it to the results I do myself
12 views

### What I do wrong with my speech recognition CTC model

I want to train an english speech to text model using architecture similar to deepspeech. In general it has 4 blocks: feature extraction I used melspectrogram. (I used n_mels=80) This translates (...
1 vote
36 views

### Handle multiple categorial features in character level RNN

I am working on a fantasy name generator and I have 2 auxiliary categorical features (gender and race). I initially tried concatenating their one hot tensors directly into the input tensor (I think it'...
29 views

### Using simple RNN to identify a simple dynamic linear system

I have been trying to identify a simple linear second order system (e.g. a pendulum or a mass-spring system), by simulating it in Python using backwards-euler method and then feeding the step changes ...
39 views

### Does a RNN also need a 3-dimensional input vector for a "Point-To-Point" forecast?

I know that for many applications a RNN (e.g. LSTM) needs a 3-dimensional input structure with [Batchsize, Sequence_Length, Features]. My question is if you also need a 3-dimensional input vector when ...
• 83
32 views

### Predict best chess move using RNNs

I am trying to do a project with AI: in which during any certain moment of a chess game i can predict, using a RNN trained on a kaggle dataset, the best possible move that i can make. I am having ...
1 vote
38 views

### Is it possible to determine the probability of each time sample to belong a certain class using gaussian distribution with Recurrent Neural Networks?

I'm trying to train a deep learning model that predicts the probability of each time sample in a two-component time series . In this case, I want the target tensor (Y) to be a probability value for ...
21 views

### Tensorflow RNN - implementing recursive layer

I am dealing with a regression problem, for which I wanted to try to use a recurrent neural network. The general setting is that I have to predict a continuous quantity starting from the evolution, in ...
67 views

### Binary classification using RNN not going beyond 50% accuracy

I am trying to find out the reason behind why my RNN network won't go beyond 50% for binary classification. My input data is of the shape: ...
171 views

### Converting a Standard LSTM RNN over to a Transformer Model

I am looking for some advice on converting my existing CNN/LSTM RNN over to a Transformer type model. This regression model takes a sliding window size of 240 rows with 33 features. It aims to ...
• 111
51 views

### Improving Wake-Word Detection Model Performance: Seeking Advice and Suggestions

I was assigned a task to train a wake-word detection model. Basically, it's a binary sequence classification model on audio samples where it should be 1 if it recognizes the wake word being said (e.g. ...
18 views

### Recurrent neural networks (RNN) and popular recurrence relationships

I am looking for a clean problem to study learning process of a RNN. I am thinking of using some number sequences that appear in mathematics and physics, like Fibonacci's, recurrence relations between ...
8 views

### Has someone designed a neural network which can select its own activation functions and/or have multiple activation functions in one model?

I'm wonder if there are any papers or implementations where a neural network has multiple activation functions in a single model (and layer), and preferably also where such activation functions ...
• 111
384 views

### Role of stateful parameter vs shuffle parameter in LSTM keras

I'm trying to make prediction on a multivariate time series using LSTM. I know stateful=True in keras LSTM means state(hidden) of each sequence, in a batch, at index i - is passed to the next batch, ...
1 vote
96 views

### Is timeseries forecasting for the next timeslot with a RNN a "Many-To-One" type application?

you often find applications that divide RNN according to their input and output data into the categories: One-To-One One-To-Many Many-To-One Many-To-Many as you can see e.g. (here https://...
• 83
1 vote
68 views

### Why my validation loss and accuracy decays over epochs?

Im trying to build 2 simple networks with cleaned dataset for tweets sentiment classification(0/1): one with all dense layers(binary bag of words) another with RNN layer(embedding layer). But it both ...
• 35
66 views

### Are there any R packages that support Deep RNNs?

I recently found an interesting paper on what it really means for a recurrent neural network (RNN) to be deep here. Depth can be added in several different ways (state to state, input to state, etc.) ...
1 vote
38 views

### ML Modeling Recommendation for Predicting Snake Encounters in Historical Journey Data

I have a dataset consisting of historical journey data where individuals travel from point A to point B. During their journeys, they may encounter varying numbers of animal sightings, including snakes....
• 11
1 vote
48 views

### LSTM for classification

I am new to neural networks and I want to use LSTM to classify the on/off state of devices based on power values. In my training dataset, I have power values, device one (0,1), and device 2 (0,1). 0 ...
• 11
1 vote
223 views

### Which preprocessing is the correct way to forecast time-series data using LSTM?

I just started to study time-series forecasting using RNN. I have a few months of time series data that was an hour unit. The data is a kind of percentage value of my little experiment and no other ...
• 113
1k views

As an exercise, I'm building a network for binary classification of sequences (whether a sequence belongs to type A or type B). The network consists of an RNN with one LSTM layer, and on top of it an ...
• 103
1 vote
2k views

### RNN with PyTorch - I don't understand the initial parameters

I would like to understand the pyTorch RNN module in detail. There I created a very simple and basic example: ...
• 21
44 views

### Confusion regarding what constitutes a feature in a LSTM?

I have a Time Series problem, where I am trying to predict a single output at time $t$, $y_t$, given the $2$ previous time steps; $X_{t-2}, X_{t-1}$. Let's just look at one observation for simplicity. ...
• 147
1 vote
793 views

### Understanding batch size, sequence, sequence length and batch length of a RNN

My Problem I'm struggling with the different definitions of batch size, sequence, sequence length and batch length of a RNN and how to use it in the correct way. First things first - let's clarify the ...
• 21
356 views

### Modeling uncertainty from known physics

I have an equation given by: $$\frac{\mathrm{d} s}{\mathrm{d} t}=4a−2s+\lambda(s)$$ where, $a$ is an input constant and $\lambda$ is a non-linear term that depends on $s$. I know that the true ...
279 views

### How many parameter in an RNN?

I came across this question asking about the number of parameters in an RNN layer, from my understanding it is the number of weights and biases, which in this case is five. Can someone confirm this? ...
860 views

### How to make an RNN model in PyTorch that has a custom hidden layer(s) and that is compatible with PackedSequence

I want to make an RNN that has for example more hidden layers or layer normalization. I know that is it possible to make a custom RNN by subclassing nn.module, but with this approach is it not ...
369 views

### Working with time series data with several times stamps on a dates, and implementing machine learning

I'm trying to implement predictive analytics on a production data. my goal is to predict next downtime, it's reason and issues. My data looks like below; ...
• 151
42 views

### Why is a RNN inherently better for Time series than normal NN?

Similar to this question but I would like further clarification. I understand that in abstract, RNNs can process inputs recursively and feed some state of memory through the recursion to have a sense ...
1 vote
153 views

### How is RNN decoder output calculated?

I was trying to read RNN Encoder Decoder paper. RNN (plain RNN i.e. non encoder-decoder RNN) It starts with giving equation for RNN: hidden state in RNN is given as: ... equation (1) where f is a ...
• 199
1 vote