Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
0 votes
1 answer
280 views

Training the document page layout and classifying good/bad layouts

I have a use case where I am supposed to get the coordinates of each block element in a page (whether its paragraph, image, table) where I train a model to understand how they are placed in a given ...
3 votes
1 answer
233 views

What are the key differences between a MLP with lagged features and a RNN

I've been working with MLP's for a while. Whenever I assumed that the past values of a feature might be useful for predicting the future values of Y, I would just create a new column in my data frame ...
2 votes
1 answer
67 views

Principles of time series analysis by neural network models

I can understand that, in the case of speech signals, words are correlated, and therefore one should have a reason to believe that RNNs or LSTMs could predict future observations by running some ...
1 vote
1 answer
254 views

How to classify a dataset containing variable size list of lists?

I have a dataset which has a list of lists as an input (each row) and the labels are in order of (0-9). The inside lists are of two lengths, 8 and 10. Each input list is of variable length ...
0 votes
1 answer
63 views

Why is my neural language model performing so poorly?

I am trying to create a word-level Haiku generator using an LSTM neural network. I am scraping haikus from Reddit's r/haiku, and wanted to start with a "simple" model: my training data is ...
0 votes
1 answer
12 views

the time complexity one2many LSTM [closed]

""Hi, Do you know the time complexity one-2many LSTM ?""
0 votes
2 answers
2k views

How to scale exponential data for a regression problem?

I understand that I should be scaling features between (0, 1) before feeding them into a neural network. However, what happens if future data could be larger than my current training data? For ...
2 votes
2 answers
126 views

Why does a filter need to be applied to the output of the input gate before cell state is added to?

In a neural network there are 4 gates: input, output, forget and a gate whose output performs element wise multiplication with the output of the input gate, which is added to the cell state (I don't ...
1 vote
1 answer
64 views

How can I convert my predictions to text after predicting using RNN?

I'm building PoS tagger for our language. I give tokens to the words and tags using Tokenizer(). Functions for word and tag are different. ...
0 votes
3 answers
302 views

How to apply a different Loss function to one specific Label?

I got a recurrent neural network in Keras, which classifies on 14 labels. The first label is the most important one and should be predicted with the highest accuracy. The other labels don't have to be ...
3 votes
1 answer
69 views

Contextual Spell Correction

I want to create a spell checker that corrects the spelling mistakes contextually. For example, Erroneous sentence: I want to apply for credit cart Corrected sentence: I want to apply for credit ...
1 vote
1 answer
650 views

Training neural network for regression with gaussian output layer

How does one train a neural network model that does regression over real values, using a gaussian output layer? ie estimating the mean and std parameters of the prediction. Since during training there ...
2 votes
1 answer
109 views

How to combine data having similar distribution?

I have a collection of time series data with data points of around 2 years of daily data. I am thinking of a way to increase the number of data points in it so that the neural network gets a better ...
2 votes
1 answer
833 views

How to represent the number of neurons in an LSTM for architecture schematic?

I'm trying to visualise a neural network schematic and found a great tool for building schematics here http://alexlenail.me/NN-SVG/index.html. I've edited the SVG file to change one of the dense ...
0 votes
1 answer
121 views

Optimal input setup for character-level text classification RNN

I want to classify 500-character long text samples as to whether they look like natural language using a character-level RNN. I'm unsure as to the best way to feed the input to the RNN. Here are two ...
0 votes
2 answers
154 views

Is the number of bidirectional LSTMs in encoder-decoder model equal to the maximum length of input text/characters?

I'm confused about this aspect of RNNs while trying to learn how seq2seq encoder-decoder works at https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/. It ...
0 votes
2 answers
82 views

What Non-linearities are best in Denoising RNN Autoencoders and where should the go?

I’m employing a denoising RNN autoencoder for a project relating to motion capture data. This is my first time using auto encoder architectures and I was just wondering what non-linearities should be ...
4 votes
1 answer
291 views

Importance/intuition behind stacking RNNs

Nowadays there's a trend towards using architectures of "deep" RNNs i.e. vertically stacked RNNs. RNN chapter from Bengio's bookThese networks seem to work well in practice. What's the intuition ...
1 vote
3 answers
464 views

Intuition behind the RNN/LSTM hidden state?

What's the intuition behind the hidden states of RNN/LSTM? Are they similar to the hidden states of HMM (Hidden Markov Model)?
3 votes
1 answer
4k views

Predicting next number in a sequence - data analysis

I am a machine learning newbie and I am working on a project where I'm given a sequence of integers all of which are in the range 0 to 70. My goal is to predict the next integer in the sequence given ...
5 votes
1 answer
1k views

Difference Between Attention and Fully Connected Layers in Deep Learning

There have been several papers in the last few years on the so-called "Attention" mechanism in deep learning (e.g. 1 2). The concept seems to be that we want the neural network to focus on ...
1 vote
1 answer
772 views

Accuracy and Loss in MLP

I am trying to explore models for predicting whether a team will win or lose based on features about the team and their opponent. My training data is 15k samples with 760 numerical features. Each ...
0 votes
1 answer
193 views

How does an RNN differ from a CBOW model

CBOW: We are trying to predict the next word based on the context (defined as a certain window of words around the target word) RNN can also be used for predicting the next word in a sequence, where ...
0 votes
1 answer
964 views

Predictions with arbitrairy sequence length for stateful RNN (LSTM/GRU) in Keras

I have time series data of the following properties: input shape: (num_timesteps, num_features) output shape: (num_timesteps, num_outputs) I reshape it to batch ...
0 votes
0 answers
15 views

Is it possible to calculate a GRU RNN in its entirety by hand on a small dataset?

I want to see whether my code works and compare it to the results I do myself
0 votes
1 answer
676 views

Word-level text generation with word embeddings – outputting a word vector instead of a probability distribution

I am currently researching the topic of text generation for my university project. I decided (ofc) to go with a RNN getting a sequence of tokens as input with a target of predicting the next token ...
1 vote
1 answer
125 views

Recurrent Neural Networks Over Multiple Documents Over Time

So in my head, I have an idea about what this architecture should look like, or at least behave, but I am having trouble implementing it. So let me describe the problem, and if anyone has an idea on ...
1 vote
1 answer
239 views

Can Batch Normalization replace tanh in RNN?

Question Can Batch Normalization (BN) be inserted in RNN after $x_t@W_{xh}$, and after $h_{t-1}@W_{hh}$ to remove $f=tanh$ and bias $b_h$? If possible, will this ...
2 votes
1 answer
351 views

Policy gradient/REINFORCE algorithm with RNN: why does this converge with SGM but not Adam?

I am working on training RNN model on caption generation with REINFORCE algorithm. I adopt self-critic strategy (see paper Self-critical Sequence Training for Image Captioning) to reduce the variance. ...
2 votes
2 answers
161 views

How do I use rnn to forecast to n periods with limited data?

So this is my 1st time trying to run a small time-series dataset through an RNN, but after a lot of searching, I haven't been able to find, 1. How I can use this to forecast to n periods ? (like in ...
0 votes
1 answer
83 views

How to use LSTM for time series data?

I've an ECG data spread over time. The duration for each data is around 3 minutes (approx 180 seconds). Each second around 200 recordings were taken. So total length for each sample is approx 36000. ...
0 votes
2 answers
792 views

Does the SimpleRNN in Keras have a hidden state, or does it just use the output value as the hidden state?

When using tf.keras.layers.SimpleRNN,does this SimpleRNN have a hidden state, or does it just use the output value as the hidden state. That is, does it follow the ...
3 votes
1 answer
79 views

Predicting t+1 from a set of sequences

Say I have have an experiment where I release a single rat into a maze and wait for it to reach the end. Say I also track this rat's position in the maze at various times. Let's do this $n$ times. Now,...
0 votes
1 answer
97 views

Encoder-Decoder performance time

I have two encoder-decoder models. *First model: *Second model: When I check the performance of the models I get approximately the same performance time (First model ~ 42 sec, Second model ~ 40 ...
22 votes
1 answer
3k views

Understanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTT

What I'm trying to do What I am trying to do is predicting the next data-point $x_t$ for each point in the timeseries $[x_0, x_1, x_2,...,x_T]$ in the context of a date-stream in real-time, in theory ...
1 vote
1 answer
203 views

CNN logarithmic path length

Path length between positions can be logarithmic when using dilated convolutions, left-padding for text Could anyone elaborate on the above statement quoted from CNN and RNN comparisons ?
0 votes
0 answers
7 views

What I do wrong with my speech recognition CTC model

I want to train an english speech to text model using architecture similar to deepspeech. In general it has 4 blocks: feature extraction I used melspectrogram. (I used n_mels=80) This translates (...
6 votes
2 answers
343 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
1 vote
1 answer
30 views

Handle multiple categorial features in character level RNN

I am working on a fantasy name generator and I have 2 auxiliary categorical features (gender and race). I initially tried concatenating their one hot tensors directly into the input tensor (I think it'...
1 vote
1 answer
153 views

What model should I use for multiple time series input

I want to predict bacteria plate count in the water from time series(around 10000 values in a row) of water temperature on a one minute granularity, and other daily climate data including min and max ...
0 votes
1 answer
103 views

What is the right Pytorch RNN implementation?

I read about RNN in pytorch: RNN — PyTorch 1.12 documentation. According to the document the RNN run the following function: I looked on another RNN example (from pytorch tutorial): NLP FROM SCRATCH: ...
0 votes
0 answers
26 views

Using simple RNN to identify a simple dynamic linear system

I have been trying to identify a simple linear second order system (e.g. a pendulum or a mass-spring system), by simulating it in Python using backwards-euler method and then feeding the step changes ...
2 votes
1 answer
1k views

Metrics for presenting RNN/LSTM result

I am working on two different architectures based on the LSTM model to predict the user's next action based on the previous actions. I am wondering, what is the best way to present the result? Is it ...
2 votes
2 answers
442 views

Timeseries LSTM: does test data need to come after training data?

I have one single, very long time series. I want to train an LSTM to distinguish between two behaviours (A or B) at every timestep (sequence-to-sequence). Because the time series is very long, I plan ...
2 votes
1 answer
207 views

What is the output of multivariate LSTM model?

I am currently trying to build an LSTM model by using multivariate inputs, but I don't understand what exact output I am predicting. I am currently using 5 features in the data as input data: ...
3 votes
1 answer
76 views

preprocessing time sequence

I have a long list of event (400 unique events, sequence ~10M long). I want to train an RNN to predict next event. The preprocessing steps i took are: (1) turning to OneHotEncoding using pandas: <...
2 votes
1 answer
90 views

What is the best way Reinforcement learning, RNN or others to predict the best action we have to take to maximize sales?

I have a dataset composed of few features : customerId, actionDay1, SalesDay1, actionDay20, SalesDay20, actionDay30, SalesDay30 action can be : call email face ...
1 vote
1 answer
660 views

LSTM model with exogenous factors

I have the following 3 columns in my dataset: 1.month, 2.day_of_week, 3.quantity. I would like to predict the future values of quantity, having following variables as explanatory: One-hot encoding of ...
1 vote
1 answer
612 views

Generalization of RNN/LSTM/GRU... model

Given a time-series prediction with a Recurrent Neural Network (doesn't matter if LSTM/GRU/...), a forecast might look like this: to_predict (orange) was fed to the model, predicted (purple) is the ...
0 votes
1 answer
1k views

ValueError: Error when checking input: expected the_input to have 3 dimensions, but got array with shape (14174, 1)

hope you're all doing good ! I am working on Automatic Speech Recognition with Python with the LibriSpeech Dataset. After preprocessing the audios data and applying an "MFCC featurizing" I ...

1
2 3 4 5
15