Skip to main content

Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
0 votes
3 answers
327 views

How to apply a different Loss function to one specific Label?

I got a recurrent neural network in Keras, which classifies on 14 labels. The first label is the most important one and should be predicted with the highest accuracy. The other labels don't have to be ...
3 votes
1 answer
72 views

Contextual Spell Correction

I want to create a spell checker that corrects the spelling mistakes contextually. For example, Erroneous sentence: I want to apply for credit cart Corrected sentence: I want to apply for credit ...
1 vote
2 answers
51 views

Why don't we gradually update the activation parameters in RNN from one activation to the next as the network is learning more?

I'm very new to (unidirectional, vanilla) RNN and sequence modeling in general, and all I understood about the motivation on having the connection between two successive hidden layers/activation is ...
2 votes
1 answer
112 views

How to combine data having similar distribution?

I have a collection of time series data with data points of around 2 years of daily data. I am thinking of a way to increase the number of data points in it so that the neural network gets a better ...
2 votes
2 answers
132 views

Why does a filter need to be applied to the output of the input gate before cell state is added to?

In a neural network there are 4 gates: input, output, forget and a gate whose output performs element wise multiplication with the output of the input gate, which is added to the cell state (I don't ...
1 vote
1 answer
821 views

Training neural network for regression with gaussian output layer

How does one train a neural network model that does regression over real values, using a gaussian output layer? ie estimating the mean and std parameters of the prediction. Since during training there ...
0 votes
0 answers
30 views

Fuzzy Name Matching with Machine Learning. Input data encoding

I have a huge dataset: Last name, first name, date of birth of Indian residents and I need to match them for similarity. The matching is fuzzy, the data looks like this (names are fictitious for the ...
2 votes
1 answer
898 views

How to represent the number of neurons in an LSTM for architecture schematic?

I'm trying to visualise a neural network schematic and found a great tool for building schematics here http://alexlenail.me/NN-SVG/index.html. I've edited the SVG file to change one of the dense ...
0 votes
0 answers
10 views

How to combine Embedding layer with 3D input and 2D input in Pytorch

This familiar with my ideas. How to use Embedding() with 3D tensor in Keras? I'm re-implementing some table-to-text papers using RNN-based seq2seq (like this one https://arxiv.org/pdf/1603.07771v3) ...
0 votes
2 answers
157 views

Is the number of bidirectional LSTMs in encoder-decoder model equal to the maximum length of input text/characters?

I'm confused about this aspect of RNNs while trying to learn how seq2seq encoder-decoder works at https://machinelearningmastery.com/configure-encoder-decoder-model-neural-machine-translation/. It ...
0 votes
1 answer
121 views

Optimal input setup for character-level text classification RNN

I want to classify 500-character long text samples as to whether they look like natural language using a character-level RNN. I'm unsure as to the best way to feed the input to the RNN. Here are two ...
0 votes
0 answers
6 views

How BPTT updating the weights while input are varing

how the RNN gets trained(BPTT) when the input size is varying because to update the weights the network has to be stable right please reply on this Thanks in advance
1 vote
1 answer
844 views

Accuracy and Loss in MLP

I am trying to explore models for predicting whether a team will win or lose based on features about the team and their opponent. My training data is 15k samples with 760 numerical features. Each ...
12 votes
1 answer
21k views

Keras LSTM with 1D time series

I'm learning how to use Keras and I've had reasonable success with my labelled dataset using the examples on Chollet's Deep Learning for Python. The data set is ~1000 Time Series with length 3125 with ...
5 votes
1 answer
2k views

Difference Between Attention and Fully Connected Layers in Deep Learning

There have been several papers in the last few years on the so-called "Attention" mechanism in deep learning (e.g. 1 2). The concept seems to be that we want the neural network to focus on ...
0 votes
1 answer
781 views

Word-level text generation with word embeddings – outputting a word vector instead of a probability distribution

I am currently researching the topic of text generation for my university project. I decided (ofc) to go with a RNN getting a sequence of tokens as input with a target of predicting the next token ...
3 votes
1 answer
4k views

Predicting next number in a sequence - data analysis

I am a machine learning newbie and I am working on a project where I'm given a sequence of integers all of which are in the range 0 to 70. My goal is to predict the next integer in the sequence given ...
0 votes
0 answers
14 views

Semantics Building In LSTM-Based Models - How does a LSTM is able to extract and represent long data using just one value (long-memory)

How does a LSTM is able to extract and represent long sequences with data while using just one value (long-memory / LM) to maintain all this information? If multiple value were used, it could be ...
0 votes
1 answer
213 views

How does an RNN differ from a CBOW model

CBOW: We are trying to predict the next word based on the context (defined as a certain window of words around the target word) RNN can also be used for predicting the next word in a sequence, where ...
22 votes
1 answer
3k views

Understanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTT

What I'm trying to do What I am trying to do is predicting the next data-point $x_t$ for each point in the timeseries $[x_0, x_1, x_2,...,x_T]$ in the context of a date-stream in real-time, in theory ...
0 votes
1 answer
967 views

Predictions with arbitrairy sequence length for stateful RNN (LSTM/GRU) in Keras

I have time series data of the following properties: input shape: (num_timesteps, num_features) output shape: (num_timesteps, num_outputs) I reshape it to batch ...
2 votes
1 answer
269 views

Can Batch Normalization replace tanh in RNN?

Question Can Batch Normalization (BN) be inserted in RNN after $x_t@W_{xh}$, and after $h_{t-1}@W_{hh}$ to remove $f=tanh$ and bias $b_h$? If possible, will this ...
1 vote
1 answer
126 views

Recurrent Neural Networks Over Multiple Documents Over Time

So in my head, I have an idea about what this architecture should look like, or at least behave, but I am having trouble implementing it. So let me describe the problem, and if anyone has an idea on ...
2 votes
2 answers
166 views

How do I use rnn to forecast to n periods with limited data?

So this is my 1st time trying to run a small time-series dataset through an RNN, but after a lot of searching, I haven't been able to find, 1. How I can use this to forecast to n periods ? (like in ...
2 votes
1 answer
369 views

Policy gradient/REINFORCE algorithm with RNN: why does this converge with SGM but not Adam?

I am working on training RNN model on caption generation with REINFORCE algorithm. I adopt self-critic strategy (see paper Self-critical Sequence Training for Image Captioning) to reduce the variance. ...
0 votes
1 answer
59 views

Data preprocessing for time series prediction

I have a dataset that has the following structure ...
6 votes
2 answers
356 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
1 vote
1 answer
766 views

LSTM model with exogenous factors

I have the following 3 columns in my dataset: 1.month, 2.day_of_week, 3.quantity. I would like to predict the future values of quantity, having following variables as explanatory: One-hot encoding of ...
1 vote
1 answer
87 views

How to use LSTM for time series data?

I've an ECG data spread over time. The duration for each data is around 3 minutes (approx 180 seconds). Each second around 200 recordings were taken. So total length for each sample is approx 36000. ...
0 votes
2 answers
881 views

Does the SimpleRNN in Keras have a hidden state, or does it just use the output value as the hidden state?

When using tf.keras.layers.SimpleRNN,does this SimpleRNN have a hidden state, or does it just use the output value as the hidden state. That is, does it follow the ...
1 vote
1 answer
207 views

CNN logarithmic path length

Path length between positions can be logarithmic when using dilated convolutions, left-padding for text Could anyone elaborate on the above statement quoted from CNN and RNN comparisons ?
-2 votes
2 answers
47 views

Determining the threshold value for the neural network

I have a dataset with last name, first name, middle name of people participating in sporting events. I need to train a neural network that will match similar surnames, first names and patronymics. But ...
0 votes
0 answers
107 views

Encoder-Decoder performance time

I have two encoder-decoder models. *First model: *Second model: When I check the performance of the models I get approximately the same performance time (First model ~ 42 sec, Second model ~ 40 ...
3 votes
1 answer
85 views

Predicting t+1 from a set of sequences

Say I have have an experiment where I release a single rat into a maze and wait for it to reach the end. Say I also track this rat's position in the maze at various times. Let's do this $n$ times. Now,...
1 vote
1 answer
160 views

What model should I use for multiple time series input

I want to predict bacteria plate count in the water from time series(around 10000 values in a row) of water temperature on a one minute granularity, and other daily climate data including min and max ...
0 votes
0 answers
17 views

What is the shape of the hidden/cell state of convLSTM2D?

I am new to convLSTM2D and I understand how it works, however, I am confused about the shape of the hidden states at different epochs ...
1 vote
0 answers
27 views

recognition of names, surnames and patronymics

is there an example of neural networks on Github or Kaggle that perform the task of recognizing identical surnames, first names and patronymics? I'm just learning neural networks so it's interesting ...
3 votes
1 answer
98 views

What Models should i try for this problem?

I need some advice for a problem i'm working on with automobile data. The vehicles provide a series of codes at every second which are bieng stored, though it can vary how many. For example , at time ...
0 votes
1 answer
114 views

What is the right Pytorch RNN implementation?

I read about RNN in pytorch: RNN — PyTorch 1.12 documentation. According to the document the RNN run the following function: I looked on another RNN example (from pytorch tutorial): NLP FROM SCRATCH: ...
2 votes
1 answer
216 views

What is the output of multivariate LSTM model?

I am currently trying to build an LSTM model by using multivariate inputs, but I don't understand what exact output I am predicting. I am currently using 5 features in the data as input data: ...
2 votes
1 answer
95 views

What is the best way Reinforcement learning, RNN or others to predict the best action we have to take to maximize sales?

I have a dataset composed of few features : customerId, actionDay1, SalesDay1, actionDay20, SalesDay20, actionDay30, SalesDay30 action can be : call email face ...
3 votes
1 answer
84 views

preprocessing time sequence

I have a long list of event (400 unique events, sequence ~10M long). I want to train an RNN to predict next event. The preprocessing steps i took are: (1) turning to OneHotEncoding using pandas: <...
2 votes
1 answer
1k views

Metrics for presenting RNN/LSTM result

I am working on two different architectures based on the LSTM model to predict the user's next action based on the previous actions. I am wondering, what is the best way to present the result? Is it ...
2 votes
2 answers
605 views

Timeseries LSTM: does test data need to come after training data?

I have one single, very long time series. I want to train an LSTM to distinguish between two behaviours (A or B) at every timestep (sequence-to-sequence). Because the time series is very long, I plan ...
0 votes
1 answer
549 views

Regression with LSTM network: use multiple time series as input

I've spent a few days on this and am starting to think I'm missing the obvious solution as this doesn't seem like a very uncommon problem. As an example dataset: I have 100 measurements with each a ...
0 votes
2 answers
92 views

Understanding Layers in Recurrent Neural Networks for NLP

In convolution neural networks, we have a concept that inner layers learn fine features like lines and edges, while outer layers learn more complex shapes. Do we have any such understanding for ...
0 votes
0 answers
10 views

Deep neural network is plateauing on a regression task

I'm training a deep neural network on temporal graph data. Currently, I'm trying to get a feel for how large / complex of a model I should aim for, so I'm trying to overfit to my smallest dataset. ...
1 vote
1 answer
81 views

Transferring the hidden state of a RNN to another RNN

I am using Reinforcement Learning to teach an AI an Austrian Card Game with imperfect information called Schnapsen. For different states of the game, I have different neural networks (which use ...
0 votes
1 answer
1k views

ValueError: Error when checking input: expected the_input to have 3 dimensions, but got array with shape (14174, 1)

hope you're all doing good ! I am working on Automatic Speech Recognition with Python with the LibriSpeech Dataset. After preprocessing the audios data and applying an "MFCC featurizing" I ...
2 votes
1 answer
57 views

Modeling Encoder-Decoder according to instructions from a paper

I am new to this field and I was reading a paper "Predicting citation counts based on deep neural network learning techniques". There the authors describe the code that they implemented if ...

1
2 3 4 5
15