2022 Developer Survey is open! Take survey.

Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
0 votes
0 answers
6 views

Time series classification but with a sequence in output

I'm using Python and I have a training set of sequences of this shape: (None, 9, 25), where: 9 are rows representing years from 2012 to 2020 25 are features. So each of this 25 features has a value ...
user avatar
1 vote
1 answer
21 views

What is the purpose of Sequence Length parameter in RNN (specifically on PyTorch)?

I am trying to understand RNN. I got a good sense of how it works on theory. But then on PyTorch you have two extra dimensions to your input data: batch size (number of batches) and sequence length. ...
user avatar
  • 13
1 vote
1 answer
20 views

RNN/LSTM timeseries, with fixed attributes per run

I have a multivariate time series of weather date: temperature, humidity and wind strength ($x_{c,t},y_{c,t},z_{c,t}$ respectively). I have this data for a dozen different cities ($c\in {c_1,c_2,...,...
user avatar
  • 13
0 votes
0 answers
8 views

What should be the loss and accuracy value while training the input and output data in deep learning using jupyter notebook? [closed]

I am working on fault detection and fault classification in power system using deep learning, when I am training the input data (fault coefficients m, n, p, q) and output data (fault type A-G, B-G, C-...
user avatar
0 votes
0 answers
21 views

PyTorch: LSTM training loss not decreasing; starting at very high loss

I am training an LSTM to give counts of the number of items in buckets. There are 252 buckets. However, I am running into an issue with very large MSELoss that does not decrease in training (meaning ...
user avatar
0 votes
0 answers
19 views

Clarification on "predict the next character given the previous 100 characters"

I am studying Justin Johnson's lecture on RNNs Lecture recording: https://www.youtube.com/watch?v=dUzLD91Sj-o&list=PL5-TkQAfAZFbzxjBHtzdVCWE0Zbhomg7r&index=12&t=3177s One of the examples ...
user avatar
  • 101
1 vote
0 answers
20 views

How to predict a mathematical progression with keras

I try the following model for a many-to-many recurrent network: ...
user avatar
0 votes
1 answer
26 views

Training data for anomaly detection using LSTM Autoencoder

I am building an time-series anomaly detection engine using LSTM autoencoder. I read this article where the author suggests to train the model on clean data only in response to a comment. However, in ...
user avatar
0 votes
0 answers
12 views

LSTM Autoencoders vs LSTM

I'm working on a time-series anomaly detection project. I have read that both LSTM Autoencoders and LSTM can do the job. Can someone please help me understand what are the advantages of each i.e. when ...
user avatar
0 votes
0 answers
12 views

How are session-parallel mini-batches used for training RNNs for session-based recommender tasks?

I am reading this paper on session-based recommenders with RNNs: https://arxiv.org/abs/1511.06939. During the training phase, the authors apply what they call "session-parallel mini-batches,"...
user avatar
0 votes
0 answers
14 views

Error while Pre-processing Audio Data using Librosa (audio analysis library in python) for DL model

I am beginner in Audio classification field in DL. I followed a YouTube Music Genre Classification Series, which is working fine and been very helpful but I have a problem/error in pre-processing part....
user avatar
1 vote
0 answers
14 views

Does N-gram language model for text generation are more efficient than Neural Network language models?

I recently build an language model with N-gram model for text generation and for change I started exploring Neural Network for text generation. One thing I observed that the previous model results ...
user avatar
  • 11
0 votes
0 answers
16 views

1D Sequence Classification using Circular Dilated Convolutional Neural Networks

I am working on a multiclass classification task on long 1D sequences. The sequence length may vary between $512$ and $512 \cdot 60$ timesteps, a slice of $100$ timesteps might look like this: What ...
user avatar
  • 143
0 votes
0 answers
8 views

What model would be best suitable for Multi-variate Binary Classification?

My main objective here is classification, either a vehicle or pedestrian The Dataset I have is as follows, this was a data I collected using Radar for a vehicle going away from a radar , its produced ...
user avatar
0 votes
2 answers
63 views

Keras: ambiguity regarding state maintenance in RNNs

The following is mentioned in the official keras RNN documentation (https://www.tensorflow.org/guide/keras/rnn). By "Normally", I assume they mean when stateful=False, which is also the ...
user avatar
  • 89
1 vote
0 answers
9 views

How are the weights defined in a (linear-chain) Conditional Random Field?

Edit: i saw that i mixed up i (in the graph) and t (in the formula), in the following i equivalent to t I am trying to understand the theory behind linear chain Conditional Random Fields. I have now ...
user avatar
  • 11
0 votes
0 answers
10 views

How to use teacher forcing in a LSTM

For my timeseries problem it seems obvious to use teacher forcing. For example in the case of predicting the new timestep in a real life scenario, I do have access to all the ground truths for all ...
user avatar
0 votes
0 answers
13 views

How to use LSTM on Human Interface Data?

I have to classify gestures using LSTM or any other neural network approach. For every time step(row), I have 34 features that belong to a gesture. Like this, some gestures correspond to a number of ...
user avatar
1 vote
1 answer
32 views

NLP LSTM input basic doubt

I have a basic doubt with regards to conversion of text to numbers and feeding it to LSTM. I am aware of the different methods such as OneHot, CountVectorizer, TfIDF, Word2vec etc. My doubt is, If we ...
user avatar
  • 53
2 votes
1 answer
38 views

Generate paragraphs from given words

I am trying to build a ML model that. will take a list of words and will try to produce sentences with those words, based on a language model on an existing corpus. Example: ...
user avatar
  • 159
1 vote
1 answer
17 views

Seq2Seq loss function

I was reading the paper neural_approach_conversational_ai.pdf. And in the section Seq2Seq for Text Generation there is a formula that i feel a bit wrong [1]: https://i.stack.imgur.com/sX0it.png Can ...
user avatar
1 vote
0 answers
7 views

Can you use both copy mechanism and BPE?

I read to alleviate the problem of Out of Vocabulary (OOV), there are two techniques: BPE Copy mechanism It appears to me they are two orthogonal approaches. Can we combine the two, i.e., we use ...
user avatar
2 votes
0 answers
21 views

1D Sequence Classification

Cross-post from https://stackoverflow.com/questions/71752744/1d-sequence-classification I am working with a long sequence (~60 000 timesteps) classification task with continuous input domain. The ...
user avatar
  • 143
0 votes
0 answers
10 views

Problems with recurrent neural net working with time steps

I am trying to design a recurrent classificatory network with Keras. I have analyzed key characteristics of the frames of a video, and from them I want to identify when certain events occur during the ...
user avatar
0 votes
0 answers
7 views

What are practical uses of MP Neurons?

Are there any practical uses of MP neurons in any industry/application or any situation where MP neuron outperforms in some metric other methods? Or is it only just used in teaching as a basis to ...
user avatar
0 votes
1 answer
18 views

Can a multilayer perceptron classify binary values?

I have a dataset in which the response variable is Sick(1) or not sick (2). As for the variables, there are a few numeric ones (2/14), all the others are variables by levels (example: 1-Abdominal pain,...
user avatar
  • 320
0 votes
0 answers
9 views

Dataset Preparation for LSTM (multiple variables)

I am struggling to conceptualize the correct way to prepare a timeseries dataset for LSTM training. My main concern is how do I train the network to 'remember' N previous steps. I have two possible ...
user avatar
0 votes
0 answers
18 views

Implementation difference between TensorFlow LSTMBlockFusedCell and PyTorch LSTM

I am trying to translate a tensorflow (version 1.14.0) LSTMBlockFusedCell module to pytorch, but I'm unable to get the same outputs for both modules with identical input and weights. PyTorch has one ...
user avatar
  • 143
1 vote
1 answer
21 views

How to classify a dataset containing variable size list of lists?

I have a dataset which has a list of lists as an input (each row) and the labels are in order of (0-9). The inside lists are of two lengths, 8 and 10. Each input list is of variable length ...
user avatar
0 votes
0 answers
20 views

Backpropagation in conventional recurrent neural network

I'm in the beginning to learn and understand recurrent neural networks. I am trying to understand the back-propagation process which helps us to find the gradients that are required to update the ...
user avatar
0 votes
0 answers
6 views

Is it possible to implement RNN with dependent and independent variables?

I wanted to implement RNN on a dataset that contains a dependent and multiple independent features. I've used ANN and DT before to do so. However, RNN seems a lot more different and doesnt focus on ...
user avatar
  • 1
0 votes
0 answers
19 views

How to classify a value in a time series using RNN?

I'm trying to classify a previous value using an RNN, I have a timeseries like this: My goal is to predict 2 points (in this example the ones in red circles), but I'm not sure how to handle this. My ...
user avatar
  • 1
0 votes
0 answers
17 views

How to build a Neural network architecture where dense layer output goes into LSTM layer input?

Hello Everyone, As you can see the above picture, I want to build a similar architecture, Can you tell me how? The CV1 and CV2 in red box are the output of a dense layer, now I have to put the output ...
user avatar
0 votes
1 answer
29 views

What are the evaluation metrics we can use for RNN models?

I'm working on a few RNN (Recurrent Neural Network) models and want to evaluate those models, so I'm looking for useful metrics to evaluate RNN models?
user avatar
0 votes
0 answers
27 views

Textbooks containing/dedicated for "NLP using PyTorch" in detail

There are some easy and comprehensive textbooks covering many deep learning concepts using PyTorch in detail. But, I am a little dissatisfied with the weightage given to RNNs compared to CNNs. I mean, ...
user avatar
  • 147
0 votes
0 answers
291 views

ValueError: Must pass 2-d input. shape=(15129, 10, 1)

I'm having a problem with reshaping a DataFrame, after doing this ...
user avatar
  • 3
0 votes
1 answer
57 views

Start & End Tokens in LSTM when making predictions

I see examples of LSTM sequence to sequence generation models which use start and end tokens for each sequence. I would like to understand when making predictions with this model, if I'd like to make ...
user avatar
  • 1
0 votes
0 answers
6 views

Does the ordering of the timesteps matter for a RNN input in Keras?

An RNN, in keras, accepts an input with the following shape ...
user avatar
0 votes
0 answers
35 views

Many to one LSTM, where some sequence values are known at prediction step

I have a time series problem, which I am modelling with an RNN (using LSTMs). The input contains a sequence of values x_0 to x_4, for predictions at positions n-k (where k is a configurable parameter -...
user avatar
0 votes
0 answers
12 views

Feed a feedforward neural net with datevariables vs rnn

I am trying to wrap my head around the pros/cons one might encounter when instead of considering a timeseries as a timeseries we break it down into singular values with the dates as variables. E.g an ...
user avatar
0 votes
1 answer
19 views

Vanishing gradients: examine output gradients

For a feedforward network or RNN, in theory we should examine the output gradients with respect to the weights over time to check whether it vanishes to zero. In my code below I am not sure whether it ...
user avatar
  • 113
0 votes
1 answer
28 views

Look ahead bias predicting a time series using features

I am making some ML methods (RF, RNN, MLP) to predict a time series value 'y' based on features 'X' and not the time series 'y' itself. My question is regarding the bias I might be including since I ...
user avatar
  • 3
0 votes
2 answers
122 views

How to set vocabulary size, padding length and embedding dimension in LSTM network?

Usually in a LSTM network, we have certain parameters that need to be set before the model can begin training. I am specifically talking about vocabulary size, padding length and embedding dimension. ...
user avatar
  • 1,241
0 votes
2 answers
112 views

Why does averaging word embedding vectors (exctracted from the NN embedding layer) work to represent sentences?

I'm puzzling to understand why the method of averaging word embeddings works in order to obtain sentence embedding, in particular considering the exercize of this post How to obtain vector ...
user avatar
0 votes
0 answers
25 views

How to pass multiple vectors to a RNN/LSTM network and get output as a vector. Can someone explain or give reference to code/text

I need to feed multiple vectors to a RNN/LSTM and get a vector as output utilizing dependencies between the vectors . How do i pass the vectors . Is there any code/reference ?
user avatar
0 votes
1 answer
22 views

Awful predictions of RNN while MSE is very low

I have encountered a strange situation where the predictions of RNN are just awful despite the fact that NN has found a minimum of loss function at 0.002 for training and 0.0013-0.0015 for validation ...
user avatar
0 votes
1 answer
17 views

What model should I use to predict a time series like this?

This series is calculated from the difference of two day's stock index. I rescaled it using sklearn's StarndardScaler. It seems LSTM does not work well on this series.
user avatar
0 votes
0 answers
7 views

Output Network in the algorithm

Can somebody explain how to understand/interpret the output network in the algorithm below? This image is taken from the article https://arxiv.org/pdf/1907.03907.pdf (3rd page).
user avatar
1 vote
0 answers
32 views

Is the RNN vanishing gradients problem really a gradients problem?

It is known that RNNs do not have long memory, that is they do not capture long dependencies. Usually this is explained by the vanishing (or explolding) gradients problem - when computing the ...
user avatar
0 votes
0 answers
29 views

Splitting sentiment analysis training data into x-train and y-train for a RNN?

Suppose I have a dataset of comments from users, around multiple websites, such that in each row, there are two comments, and one is considered more 'negative' and one more 'positive' indicated by the ...
user avatar
  • 133

1
2 3 4 5
15