Skip to main content

Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
1 vote
3 answers
2k views

What are the evaluation metrics we can use for RNN models?

I'm working on a few RNN (Recurrent Neural Network) models and want to evaluate those models, so I'm looking for useful metrics to evaluate RNN models?
Abdul Rehman's user avatar
1 vote
1 answer
11k views

ValueError: Must pass 2-d input. shape=(15129, 10, 1)

I'm having a problem with reshaping a DataFrame, after doing this ...
WojKie's user avatar
  • 23
2 votes
2 answers
2k views

Start & End Tokens in LSTM when making predictions

I see examples of LSTM sequence to sequence generation models which use start and end tokens for each sequence. I would like to understand when making predictions with this model, if I'd like to make ...
eknagi's user avatar
  • 21
0 votes
1 answer
43 views

Vanishing gradients: examine output gradients

For a feedforward network or RNN, in theory we should examine the output gradients with respect to the weights over time to check whether it vanishes to zero. In my code below I am not sure whether it ...
siegfried's user avatar
  • 113
0 votes
1 answer
132 views

Look ahead bias predicting a time series using features

I am making some ML methods (RF, RNN, MLP) to predict a time series value 'y' based on features 'X' and not the time series 'y' itself. My question is regarding the bias I might be including since I ...
JmML's user avatar
  • 3
0 votes
2 answers
4k views

How to set vocabulary size, padding length and embedding dimension in LSTM network?

Usually in a LSTM network, we have certain parameters that need to be set before the model can begin training. I am specifically talking about vocabulary size, padding length and embedding dimension. ...
spectre's user avatar
  • 2,115
1 vote
2 answers
4k views

Why does averaging word embedding vectors (exctracted from the NN embedding layer) work to represent sentences?

I'm puzzling to understand why the method of averaging word embeddings works in order to obtain sentence embedding, in particular considering the exercize of this post How to obtain vector ...
HelpNeederStudent's user avatar
0 votes
1 answer
133 views

Awful predictions of RNN while MSE is very low

I have encountered a strange situation where the predictions of RNN are just awful despite the fact that NN has found a minimum of loss function at 0.002 for training and 0.0013-0.0015 for validation ...
Puchatek Kubuś's user avatar
0 votes
1 answer
26 views

What model should I use to predict a time series like this?

This series is calculated from the difference of two day's stock index. I rescaled it using sklearn's StarndardScaler. It seems LSTM does not work well on this series.
user900476's user avatar
1 vote
0 answers
49 views

Is the RNN vanishing gradients problem really a gradients problem?

It is known that RNNs do not have long memory, that is they do not capture long dependencies. Usually this is explained by the vanishing (or explolding) gradients problem - when computing the ...
Amit Keinan's user avatar
0 votes
0 answers
125 views

Splitting sentiment analysis training data into x-train and y-train for a RNN?

Suppose I have a dataset of comments from users, around multiple websites, such that in each row, there are two comments, and one is considered more 'negative' and one more 'positive' indicated by the ...
sangstar's user avatar
  • 133
0 votes
1 answer
783 views

Word-level text generation with word embeddings – outputting a word vector instead of a probability distribution

I am currently researching the topic of text generation for my university project. I decided (ofc) to go with a RNN getting a sequence of tokens as input with a target of predicting the next token ...
czypsu's user avatar
  • 101
1 vote
0 answers
38 views

How to predict data from sequence of sequences of variable size?

input data ...
Emil's user avatar
  • 308
2 votes
0 answers
29 views

Dual Branch Recurrent Neural Network, what is the correct architecture and can I turn off one branch?

Let's suppose I want to predict the daily consumption of apples in the next 30 days of a person and I have, as input, the historical information about the past 60 days and the demographic information ...
oettam_oisolliv's user avatar
1 vote
0 answers
690 views

LSTM with variable time steps

I'm reading this post that describes how to train LSTMs with variable time step lengths. But does that have repercussions? Should I preprocess the time series in to varying permutations? e.g. Should ...
bli00's user avatar
  • 111
1 vote
1 answer
824 views

Training neural network for regression with gaussian output layer

How does one train a neural network model that does regression over real values, using a gaussian output layer? ie estimating the mean and std parameters of the prediction. Since during training there ...
Sid's user avatar
  • 677
1 vote
0 answers
176 views

Is an output layer with 2 units and softmax better than one with 1 unit and sigmoid for binary classification using LSTM?

I am using an LSTM for binary classification and initially tried a model with 1 unit in the output(Dense) layer with sigmoid as the activation function. However, it didn't perform well and I saw a few ...
Saket Vempaty's user avatar
0 votes
2 answers
901 views

How to predict a certain time span into the future with recurrent neural networks in Keras

I have the following code for time series predictions with RNNs and I would like to know whether for the testing I predict one day in advance: ...
PeterBe's user avatar
  • 83
1 vote
1 answer
81 views

Transferring the hidden state of a RNN to another RNN

I am using Reinforcement Learning to teach an AI an Austrian Card Game with imperfect information called Schnapsen. For different states of the game, I have different neural networks (which use ...
Babypopo's user avatar
1 vote
0 answers
27 views

How to model prior informaton in sequential models?

Are there any approaches to model prior information in sequential models? Such as in Sequence classification. For example, I have an input sequence [[Z, 0, 1], [Y, 1, 1]]. I need to classfy this into ...
Justin's user avatar
  • 29
2 votes
0 answers
32 views

Models for Long-Term Time-Series Forecasting and Pattern Recognition

I'm trying to find a solution for long-term electricity hourly prices forecasting. Explaining simply, I have some data from 2018 - 2021 containing Demand, Renewable Generation, Hydropower Generation, ...
Ircb's user avatar
  • 21
1 vote
2 answers
2k views

Please explain Transformer vs LSTM using a sequence prediction example

I don't understand the difference in mechanics of a transformer vs LSTM for a sequence prediction problem. Here is what I have gathered so far: LSTM: suppose we want to predict the remaining tokens ...
huy's user avatar
  • 147
0 votes
0 answers
556 views

Is is possible to make a text generator with sklearn?

So recently I made a Tensorflow model using RNN (Recurrent neural networks) and I was wondering if it was possible with sklearn too, through the usage of SVMs or Naive bayes. I searched up many ...
Aryan's user avatar
  • 101
0 votes
1 answer
21 views

Why is sequence prediction always the objective in RNN and LSTM like algorithms

The title is pretty much my question. I haven't seen any literature yet that uses a different training objective. The goal is to find the hidden states eventually, then why is it that only 1 method is ...
huy's user avatar
  • 147
0 votes
0 answers
1k views

Is it always beneficial to use return_sequences=True for time series prediction with RNN?

I roughly understand what return_sequences=True does when being used for time series prediction with RNN (each RNN cell outputs its hidden state). Now my question ...
PeterBe's user avatar
  • 83
0 votes
1 answer
213 views

How does an RNN differ from a CBOW model

CBOW: We are trying to predict the next word based on the context (defined as a certain window of words around the target word) RNN can also be used for predicting the next word in a sequence, where ...
huy's user avatar
  • 147
1 vote
1 answer
766 views

LSTM model with exogenous factors

I have the following 3 columns in my dataset: 1.month, 2.day_of_week, 3.quantity. I would like to predict the future values of quantity, having following variables as explanatory: One-hot encoding of ...
stupidwebsite's user avatar
0 votes
1 answer
3k views

Whats the minimum size sample required for a LSTM RNN model? [closed]

I have a data set of 100 rows x 100 to 300 columns. Will an LSTM RNN model work for my data or do I need more data? If my sample size is a problem are there other RNN architectures capable of modeling ...
Dean's user avatar
  • 3
0 votes
1 answer
1k views

How to eliminate Non-Trainable params in Deep Learning [closed]

First of all, I would like to know what is the cause of Non-Trainable parameters? Secondly, how do you eliminate them? I used a combined CNN-RNN, it returned that 130 Non-Trainable parameters. Thank ...
Mimi's user avatar
  • 45
0 votes
1 answer
131 views

Very low error during training of a RNN for forecasting but high test error

I use a Recurrent Neural Network for time series forecasting of electrical load data from a cooling device based on past values of the load time series and temperature values. I first normalize the ...
PeterBe's user avatar
  • 83
0 votes
1 answer
137 views

ML algorithm for high dimensional time series forecasting

I'm trying to make a forecasting model for goods prices in an economy (trying to forecast inflation). Dataset: has 300 goods prices % monthly variations for last 6 years. And also added $n$ ...
Oliver Mohr Bonometti's user avatar
0 votes
1 answer
27 views

Can anyone interpret this Recurrent Network Encoder-Decoder question?

I'm trying to earn some extra credit, so the professor won't elaborate further on what's being asked in this question: The dataset that we're given is a line-by-line file of protein sequences (...
lizardmonkey's user avatar
0 votes
1 answer
363 views

Model layer getting random two input instead of 1 input

I am running the code mentioned at link of the code Here is the code: ...
NIrbhay Mathur's user avatar
0 votes
1 answer
1k views

ValueError: Layer model expects 2 input(s), but it received 3 input tensors using generator

I am trying to fit a model using generator function and I get the following error: ...
manix velu's user avatar
1 vote
0 answers
221 views

Is it possible to combine cnn and rnn?

I would like to know if it is possible to combine rnn and cnn. I explain you : I have pictures of bikes, cars and moto and every pictures is linked to a text. For instance for a car I can have the ...
Adam's user avatar
  • 11
1 vote
2 answers
4k views

MY lstm has a really low accuracy, is there anyway to improve it?

I am trying to make a model to classify whether these patients can be diagnosed with dementia by their 35 days of biometric data. A brief summary of a dataset is below. as an input X_train data, it ...
JaeHyun's user avatar
  • 11
1 vote
0 answers
117 views

In Keras, how to organize multiple input features using pre-trained embedding mapping?

Let's say the goal is to predict weather given multiple features (temp, humidity) in the past 3 days. weather (y) can be: Sunny, Cloudy, Rainy. Temp (X1) can be : Hot, Cool, Cold. Humidity (X2) can be:...
user6304430's user avatar
2 votes
0 answers
46 views

Why do RNNs share weight?

If weights are not shared the number of parameters will be extremely large and difficult to compute which I understand. I don't understand the argument that varying length inputs are taken care of by ...
Math zombie's user avatar
1 vote
0 answers
24 views

Machine Learning for analyzing and generating sentences from given text inputs

I'm trying to create a program that will translate Sign Language to Text and apply NLP so that the text is understandable to human. I've used CNN for recognizing sign language but I don't know how to ...
Lord Dickenstein's user avatar
0 votes
1 answer
65 views

Why is my neural language model performing so poorly?

I am trying to create a word-level Haiku generator using an LSTM neural network. I am scraping haikus from Reddit's r/haiku, and wanted to start with a "simple" model: my training data is ...
Brian Barry's user avatar
1 vote
0 answers
136 views

Why we shift target(output) by one offset in language modelling

I have been working in sequence prediction tasks (very similar to language modelling) where I want to predict the next token(s)/item(s) given past sequence of tokens. I have always taken an approach ...
A.B's user avatar
  • 336
2 votes
0 answers
260 views

Applying GradCam to video classification models

In the original paper, it says that GradCam visualization can be applied to any convolution based model. The problem is stated for convolutions that process images. In my case, I am classifying videos ...
Iván Mindlin's user avatar
1 vote
1 answer
76 views

In what way is recurrent neural network state "hidden"?

Recurrent neural networks have hidden state denoted $h$. Why is the state considered "hidden"? It's clear to me what purpose the state itself serves, but I can't figure out why it is "...
Gaslight Deceive Subvert's user avatar
0 votes
2 answers
882 views

Does the SimpleRNN in Keras have a hidden state, or does it just use the output value as the hidden state?

When using tf.keras.layers.SimpleRNN,does this SimpleRNN have a hidden state, or does it just use the output value as the hidden state. That is, does it follow the ...
Ryan's user avatar
  • 31
1 vote
1 answer
44 views

How can I predict the last element of the fixed length=8 sequence after I get each element?

There are fixed length lists [X1, X2, X3, X4, X5, X6, X7, X8] like this. I have many lists like them from the past. In the future, I will get new element of current list on weekly bases. one new ...
Roman Kutubidze's user avatar
0 votes
1 answer
23 views

Normalization of possibly not fully representative data

I am trying to train a classification RNN model on a sequence of table medical data, but I stuck with the normalization problem. I realized that I cannot simply use MinMaxScaler, because of 3 problems:...
banderlog013's user avatar
0 votes
0 answers
24 views

Encoding entities with features of continuous values

Given a set of entities, I would like to predict the next in the sequence; for this purpose, I would like to use RNN. However, my first challenge is how to model the entities. A possible input ...
Dr. Strangelove's user avatar
1 vote
0 answers
98 views

One Year Ahead Forecasting with Unevenly Spaced Time Series

I have many products in my warehouses which can be "demanded" any day by my different clients. I want to forecast how many of each item will be demanded for the whole next year. Naturally, ...
Lucas's user avatar
  • 33
1 vote
1 answer
41 views

How to implement sequence to sequence models?

I have a dataset with patient demographics, diagnosis history, hospital visit dates, drugs consumed etc. All these events have time stamp information (except static info like demographics such gender, ...
The Great's user avatar
  • 2,575
1 vote
1 answer
42 views

Can RNN be replaced with non-recurrent classifier for Sequence Classification problem?

Setup: We have sequence of events that are not evenly spaced (not a time series). Length of the sequence is constant. Goal: Predict class of the event that is most probable to follow this sequence. ...
Deil's user avatar
  • 193

1 2
3
4 5
15