Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
1 vote
1 answer
22 views

Difference Between Attention and Fully Connected Layers in Deep Learning

There have been several papers in the last few years on the so-called "Attention" mechanism in deep learning (e.g. 1 2). The concept seems to be that we want the neural network to focus on ...
user avatar
  • 566
0 votes
1 answer
43 views

Multivariate timeseries classification for each group in a dataset

Let's say, I have the following dataset: ...
user avatar
0 votes
0 answers
4 views

How To Shuffle Long-Short-Term-Memory Or Gated-Recurrent-Unit Layer Cells Operation?

As you know these types of layers operate side-to-side, and although could have been implemented in Bidirectional layer to operate on both forward and backward directions. But is it possible to change ...
user avatar
0 votes
0 answers
14 views

Validation error approaches the same value for many hyperparameters

I am using kerastuner to explore the parameter space of my RNN. The validation MSE for each model seems to follow the same trend: completely level at ~0.5, a major drop around the third epoch, then ...
user avatar
1 vote
1 answer
34 views

Why do RNN text generation models treat word prediction as a classification task?

In many of the sources I have found regarding text generation with word-based RNN models (LSTM or GRU), the model is trained to perform a classification task across the vocabulary (such as with ...
user avatar
  • 113
0 votes
0 answers
17 views

Temporal rows selection for Recurrent Neural Networks

I have a time serie $x_{1},...,x_{n}$ with a temporal step $\Delta = date(x_{i+1}) - date(x_{i}) = (i+1) - (i)= 1 \ day $. For each $i \in [\![ 1,n ]\!] $, I know that the value of $x_{i}$ depends ...
user avatar
0 votes
1 answer
18 views

RNN basic doubt

Suppose if I have 2 sentences: "My name is Alex" "Alex is my name" If I am using a RNN, After processing both the sentences, Will the final output vector be the same? Because RNN ...
user avatar
  • 109
0 votes
0 answers
9 views

Machine Learning Model to Translate an Input Time Series to a Target Time Series?

I want to train a machine learning model to translate input time series signals into target (ground truth) time series signals. I have thousands of input-target training pairs similar to the ones ...
user avatar
0 votes
0 answers
12 views

Is this article applying a wrong validation concept?

About this article Short-Term Photovoltaic Power Forecasting Based on VMD and ISSA-GRU Am I right in concluding that their proposal violates the neural network basic validation concepts by proposing ...
user avatar
  • 1
0 votes
1 answer
28 views

What is the right Pytorch RNN implementation?

I read about RNN in pytorch: RNN — PyTorch 1.12 documentation. According to the document the RNN run the following function: I looked on another RNN example (from pytorch tutorial): NLP FROM SCRATCH: ...
user avatar
0 votes
0 answers
14 views

Multilayer/deep recurrent layer

I might be missing something, but I'm completely unable to find any reference about this topic. In the literature, there are many references about RNN, GRU, LSTM, STAR and many other architecture that ...
user avatar
1 vote
1 answer
46 views

Why can't I reproduce my results in keras using random seed?

I was doing a task using RNN to predict a time series movement. I want to make my results reproducible. So I strictly followed this post: https://stackoverflow.com/questions/32419510/how-to-get-...
user avatar
0 votes
0 answers
41 views

Why is LSTM so much more popular than GRU?

I read some papers and most of them have only used LSTM. I am doing a project using neural networks to predict stock movement. However, according to a friend of mine who is in the quant trading ...
user avatar
0 votes
0 answers
8 views

How to include current and past feature for prediction in RNN

I have a timeseries (sales for e.g.) and a correlated feature that impacts the timeseries. I want to reshape data so that I can also use current value of correlated feature as an input. I illustrate ...
user avatar
0 votes
0 answers
11 views

How to organize data to use a recurrent neural network LSTM?

I am doing an internship in bailiff society. I have to create an IA model which can improve actions to perform, based on existing timeline of actions. I've already tryed some solutions (which did not ...
user avatar
  • 1
0 votes
1 answer
14 views

How to deal with different amounts of data every day?

I am doing a time series prediction task. There are different amounts of news headlines every day, and the goal is a binary prediction task to predict next day's stock movement. The amount of ...
user avatar
1 vote
1 answer
30 views

Weather impact on plant growth

I have a data set that includes the following and am using it to learn more about data science. I have googled a bunch - but can't seem to find any examples on what I am trying to do. I am trying to ...
user avatar
0 votes
0 answers
60 views

LSTM - Why is a sliding window so important in this problem?

I've been having performance issues with my LSTM implementation. Whenever I use a sliding window, the performance seems to get better. Moreover the size of the window seems to have a large impact on ...
user avatar
  • 13
0 votes
0 answers
41 views

Help with Time Series prediction

I'm a complete n00b to both this stackexchange and ML so please don't flame me too bad. I am trying to make a prediction from Time Series data. I have about 10 years worth of 1-minute resolution price ...
user avatar
  • 101
1 vote
0 answers
13 views

RNN to model DNA sequencing classification

I have a DNA sequence dataset each mapped to a certain class. e,g TCAGCCGAGAGCTCATCGATCGTACGT 2 ATGCAGTGCATCGATCGATCGTAGAAC 3 Where the number after the sequence specifies the type of protein this ...
user avatar
0 votes
0 answers
15 views

Time series classification but with a sequence in output

I'm using Python and I have a training set of sequences of this shape: (None, 9, 25), where: 9 are rows representing years from 2012 to 2020 25 are features. So each of this 25 features has a value ...
user avatar
1 vote
1 answer
101 views

What is the purpose of Sequence Length parameter in RNN (specifically on PyTorch)?

I am trying to understand RNN. I got a good sense of how it works on theory. But then on PyTorch you have two extra dimensions to your input data: batch size (number of batches) and sequence length. ...
user avatar
  • 13
1 vote
1 answer
24 views

RNN/LSTM timeseries, with fixed attributes per run

I have a multivariate time series of weather date: temperature, humidity and wind strength ($x_{c,t},y_{c,t},z_{c,t}$ respectively). I have this data for a dozen different cities ($c\in {c_1,c_2,...,...
user avatar
  • 13
0 votes
0 answers
142 views

PyTorch: LSTM training loss not decreasing; starting at very high loss

I am training an LSTM to give counts of the number of items in buckets. There are 252 buckets. However, I am running into an issue with very large MSELoss that does not decrease in training (meaning ...
user avatar
0 votes
0 answers
22 views

Clarification on "predict the next character given the previous 100 characters"

I am studying Justin Johnson's lecture on RNNs Lecture recording: https://www.youtube.com/watch?v=dUzLD91Sj-o&list=PL5-TkQAfAZFbzxjBHtzdVCWE0Zbhomg7r&index=12&t=3177s One of the examples ...
user avatar
  • 101
1 vote
0 answers
22 views

How to predict a mathematical progression with keras

I try the following model for a many-to-many recurrent network: ...
user avatar
0 votes
1 answer
48 views

Training data for anomaly detection using LSTM Autoencoder

I am building an time-series anomaly detection engine using LSTM autoencoder. I read this article where the author suggests to train the model on clean data only in response to a comment. However, in ...
user avatar
0 votes
0 answers
26 views

LSTM Autoencoders vs LSTM

I'm working on a time-series anomaly detection project. I have read that both LSTM Autoencoders and LSTM can do the job. Can someone please help me understand what are the advantages of each i.e. when ...
user avatar
0 votes
0 answers
16 views

How are session-parallel mini-batches used for training RNNs for session-based recommender tasks?

I am reading this paper on session-based recommenders with RNNs: https://arxiv.org/abs/1511.06939. During the training phase, the authors apply what they call "session-parallel mini-batches,"...
user avatar
0 votes
0 answers
30 views

Error while Pre-processing Audio Data using Librosa (audio analysis library in python) for DL model

I am beginner in Audio classification field in DL. I followed a YouTube Music Genre Classification Series, which is working fine and been very helpful but I have a problem/error in pre-processing part....
user avatar
1 vote
0 answers
22 views

Does N-gram language model for text generation are more efficient than Neural Network language models?

I recently build an language model with N-gram model for text generation and for change I started exploring Neural Network for text generation. One thing I observed that the previous model results ...
user avatar
  • 11
0 votes
0 answers
25 views

1D Sequence Classification using Circular Dilated Convolutional Neural Networks

I am working on a multiclass classification task on long 1D sequences. The sequence length may vary between $512$ and $512 \cdot 60$ timesteps, a slice of $100$ timesteps might look like this: What ...
user avatar
  • 153
0 votes
0 answers
9 views

What model would be best suitable for Multi-variate Binary Classification?

My main objective here is classification, either a vehicle or pedestrian The Dataset I have is as follows, this was a data I collected using Radar for a vehicle going away from a radar , its produced ...
user avatar
0 votes
2 answers
69 views

Keras: ambiguity regarding state maintenance in RNNs

The following is mentioned in the official keras RNN documentation (https://www.tensorflow.org/guide/keras/rnn). By "Normally", I assume they mean when stateful=False, which is also the ...
user avatar
  • 89
1 vote
0 answers
16 views

How are the weights defined in a (linear-chain) Conditional Random Field?

Edit: i saw that i mixed up i (in the graph) and t (in the formula), in the following i equivalent to t I am trying to understand the theory behind linear chain Conditional Random Fields. I have now ...
user avatar
  • 11
0 votes
0 answers
15 views

How to use teacher forcing in a LSTM

For my timeseries problem it seems obvious to use teacher forcing. For example in the case of predicting the new timestep in a real life scenario, I do have access to all the ground truths for all ...
user avatar
0 votes
0 answers
13 views

How to use LSTM on Human Interface Data?

I have to classify gestures using LSTM or any other neural network approach. For every time step(row), I have 34 features that belong to a gesture. Like this, some gestures correspond to a number of ...
user avatar
1 vote
1 answer
51 views

NLP LSTM input basic doubt

I have a basic doubt with regards to conversion of text to numbers and feeding it to LSTM. I am aware of the different methods such as OneHot, CountVectorizer, TfIDF, Word2vec etc. My doubt is, If we ...
user avatar
  • 109
2 votes
1 answer
66 views

Generate paragraphs from given words

I am trying to build a ML model that. will take a list of words and will try to produce sentences with those words, based on a language model on an existing corpus. Example: ...
user avatar
  • 159
1 vote
1 answer
29 views

Seq2Seq loss function

I was reading the paper neural_approach_conversational_ai.pdf. And in the section Seq2Seq for Text Generation there is a formula that i feel a bit wrong [1]: https://i.stack.imgur.com/sX0it.png Can ...
user avatar
1 vote
0 answers
7 views

Can you use both copy mechanism and BPE?

I read to alleviate the problem of Out of Vocabulary (OOV), there are two techniques: BPE Copy mechanism It appears to me they are two orthogonal approaches. Can we combine the two, i.e., we use ...
user avatar
2 votes
0 answers
41 views

1D Sequence Classification

Cross-post from https://stackoverflow.com/questions/71752744/1d-sequence-classification I am working with a long sequence (~60 000 timesteps) classification task with continuous input domain. The ...
user avatar
  • 153
0 votes
0 answers
11 views

Problems with recurrent neural net working with time steps

I am trying to design a recurrent classificatory network with Keras. I have analyzed key characteristics of the frames of a video, and from them I want to identify when certain events occur during the ...
user avatar
0 votes
0 answers
8 views

What are practical uses of MP Neurons?

Are there any practical uses of MP neurons in any industry/application or any situation where MP neuron outperforms in some metric other methods? Or is it only just used in teaching as a basis to ...
user avatar
0 votes
1 answer
21 views

Can a multilayer perceptron classify binary values?

I have a dataset in which the response variable is Sick(1) or not sick (2). As for the variables, there are a few numeric ones (2/14), all the others are variables by levels (example: 1-Abdominal pain,...
user avatar
  • 302
0 votes
0 answers
11 views

Dataset Preparation for LSTM (multiple variables)

I am struggling to conceptualize the correct way to prepare a timeseries dataset for LSTM training. My main concern is how do I train the network to 'remember' N previous steps. I have two possible ...
user avatar
0 votes
0 answers
24 views

Implementation difference between TensorFlow LSTMBlockFusedCell and PyTorch LSTM

I am trying to translate a tensorflow (version 1.14.0) LSTMBlockFusedCell module to pytorch, but I'm unable to get the same outputs for both modules with identical input and weights. PyTorch has one ...
user avatar
  • 153
1 vote
1 answer
32 views

How to classify a dataset containing variable size list of lists?

I have a dataset which has a list of lists as an input (each row) and the labels are in order of (0-9). The inside lists are of two lengths, 8 and 10. Each input list is of variable length ...
user avatar
0 votes
0 answers
21 views

Backpropagation in conventional recurrent neural network

I'm in the beginning to learn and understand recurrent neural networks. I am trying to understand the back-propagation process which helps us to find the gradients that are required to update the ...
user avatar
0 votes
0 answers
12 views

Is it possible to implement RNN with dependent and independent variables?

I wanted to implement RNN on a dataset that contains a dependent and multiple independent features. I've used ANN and DT before to do so. However, RNN seems a lot more different and doesnt focus on ...
user avatar
  • 1

1
2 3 4 5
15