Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
0
votes
0answers
15 views

Very low error during training of a RNN for forecasting but high test error

I use a Recurrent Neural Network for time series forecasting of electrical load data from a cooling device based on past values of the load time series and temperature values. I first normalize the ...
0
votes
1answer
25 views

ML algorithm for high dimensional time series forecasting

I'm trying to make a forecasting model for goods prices in an economy (trying to forecast inflation). Dataset: has 300 goods prices % monthly variations for last 6 years. And also added $n$ ...
0
votes
1answer
19 views

Can anyone interpret this Recurrent Network Encoder-Decoder question?

I'm trying to earn some extra credit, so the professor won't elaborate further on what's being asked in this question: The dataset that we're given is a line-by-line file of protein sequences (...
0
votes
0answers
20 views

Recurrent models for asynchronous / mixed frequency time series

What are some of the RNN/LSTM models for handling mixed frequency/asynchronous time series data, such as macroeconomics, financial, precipitation, etc.? So far I have found phased lstm from a similar ...
0
votes
1answer
23 views

Model layer getting random two input instead of 1 input

I am running the code mentioned at link of the code Here is the code: ...
1
vote
0answers
17 views

how to calculate parameters of an RNN using backpropagation

I'm trying to find out the two binary inputs are identical or not using RNN. my architecture is like this: I have the following functions: Where vT is the transpose of vector v and the activation ...
0
votes
0answers
48 views

ValueError: Layer model expects 2 input(s), but it received 3 input tensors using generator

I am trying to fit a model using generator function and I get the following error: ...
1
vote
0answers
29 views

Is it possible to combine cnn and rnn?

I would like to know if it is possible to combine rnn and cnn. I explain you : I have pictures of bikes, cars and moto and every pictures is linked to a text. For instance for a car I can have the ...
1
vote
0answers
16 views

MY lstm has a really low accuracy, is there anyway to improve it?

I am trying to make a model to classify whether these patients can be diagnosed with dementia by their 35 days of biometric data. A brief summary of a dataset is below. as an input X_train data, it ...
1
vote
0answers
10 views

In Keras, how to organize multiple input features using pre-trained embedding mapping?

Let's say the goal is to predict weather given multiple features (temp, humidity) in the past 3 days. weather (y) can be: Sunny, Cloudy, Rainy. Temp (X1) can be : Hot, Cool, Cold. Humidity (X2) can be:...
2
votes
0answers
29 views

Why do RNNs share weight?

If weights are not shared the number of parameters will be extremely large and difficult to compute which I understand. I don't understand the argument that varying length inputs are taken care of by ...
0
votes
0answers
21 views

How to walk forward an LSTM autoencoder by n timesteps?

I am able to fit this autoencoder to my sequence in order to reconstruct it. However, how would I be able to walk this forward 3 timesteps to get [[11.0], [12.0], [13.0]]? ...
0
votes
0answers
9 views

Vector to Sequence RNN

Does any one have any links to any implementations of a vector to sequence model? Preferably not in the domain of image captioning.
0
votes
0answers
12 views

encoding 100,000s of sparse, binary features at each time step of RNN

I’m looking for some pointers on efficiency. I have potentially 100,000s of binary variables that i wish to encode in each time step of an RNN for binary classification of the entire sequence, but I ...
1
vote
0answers
23 views

Machine Learning for analyzing and generating sentences from given text inputs

I'm trying to create a program that will translate Sign Language to Text and apply NLP so that the text is understandable to human. I've used CNN for recognizing sign language but I don't know how to ...
0
votes
1answer
32 views

Why is my neural language model performing so poorly?

I am trying to create a word-level Haiku generator using an LSTM neural network. I am scraping haikus from Reddit's r/haiku, and wanted to start with a "simple" model: my training data is ...
1
vote
0answers
23 views

Why we shift target(output) by one offset in language modelling

I have been working in sequence prediction tasks (very similar to language modelling) where I want to predict the next token(s)/item(s) given past sequence of tokens. I have always taken an approach ...
1
vote
0answers
11 views

Applying GradCam to video classification models

In the original paper, it says that GradCam visualization can be applied to any convolution based model. The problem is stated for convolutions that process images. In my case, I am classifying videos ...
1
vote
1answer
23 views

In what way is recurrent neural network state “hidden”?

Recurrent neural networks have hidden state denoted $h$. Why is the state considered "hidden"? It's clear to me what purpose the state itself serves, but I can't figure out why it is "...
0
votes
0answers
13 views

Does the SimpleRNN in Keras have a hidden state, or does it just use the output value as the hidden state?

When using tf.keras.layers.SimpleRNN,does this SimpleRNN have a hidden state, or does it just use the output value as the hidden state. That is, does it follow the ...
0
votes
0answers
17 views

How to use gradient checkpointing on packed sequence RNN

I have a batch of sequences that have a variable length. To save computation I used pack_padded_sequence as following: ...
1
vote
1answer
10 views

How can I predict the last element of the fixed length=8 sequence after I get each element?

There are fixed length lists [X1, X2, X3, X4, X5, X6, X7, X8] like this. I have many lists like them from the past. In the future, I will get new element of current list on weekly bases. one new ...
0
votes
0answers
14 views

multivariate biLSTM text classification

I have made a model that takes vacancy data and classifies it with (bi)LSTM. The variables of the initial dataset are: positiontitle ...
0
votes
1answer
12 views

Normalization of possibly not fully representative data

I am trying to train a classification RNN model on a sequence of table medical data, but I stuck with the normalization problem. I realized that I cannot simply use MinMaxScaler, because of 3 problems:...
0
votes
0answers
16 views

Encoding entities with features of continuous values

Given a set of entities, I would like to predict the next in the sequence; for this purpose, I would like to use RNN. However, my first challenge is how to model the entities. A possible input ...
1
vote
0answers
32 views

One Year Ahead Forecasting with Unevenly Spaced Time Series

I have many products in my warehouses which can be "demanded" any day by my different clients. I want to forecast how many of each item will be demanded for the whole next year. Naturally, ...
2
votes
1answer
28 views

How to implement sequence to sequence models?

I have a dataset with patient demographics, diagnosis history, hospital visit dates, drugs consumed etc. All these events have time stamp information (except static info like demographics such gender, ...
0
votes
0answers
16 views

Keras RNN(LSTM) Loss Not Decreasing

I'm running an LSTM model to classify astrophysics time series data. I am attempting to classify each time series as either a black hole (1) or not a black hole (0). When I run my RNN, the loss is not ...
1
vote
1answer
29 views

Can RNN be replaced with non-recurrent classifier for Sequence Classification problem?

Setup: We have sequence of events that are not evenly spaced (not a time series). Length of the sequence is constant. Goal: Predict class of the event that is most probable to follow this sequence. ...
0
votes
0answers
15 views

Neural Networks with a list of undetermined length as input

I have a list of strings and for each input (each list), there is a target that is again a string. The idea is to let the network learn to generalize from the inputs. Here is one example: ...
0
votes
0answers
32 views

Modelling feature interactions in LSTM network

I created a binary model from 28 features, each sequence is 10 samples long. I tried these two models: ...
0
votes
0answers
8 views

Time series forecasting when one of the series is known

I have a problem where there are two time series $\{x_t\}_{t \geq 1}$ and $\{z_t\}_{t \geq 1}$. These two time series are correlated for fixed time instant but uncorrelated with each other across time....
1
vote
1answer
50 views

LSTM for Stock Return Prediction

I am writing my masters thesis and am using LSTMs for daily stock return prediction. So far I am only predicting numerical values but will soon explore a classification style problem and predict ...
3
votes
1answer
36 views

Recurrent Neural Network (RNN) Vanishing gradient problem - Why does it affect earlier timesteps more?

I understand the concept of backpropagation in standard neural networks and backpropagation through time with RNNs, why this causes exponentially smaller gradients at earlier time steps and most of ...
0
votes
0answers
17 views

Trying to extend this code to include additional feature volume (in addition to adj close) RNN to predict adj close

I read this article on medium https://medium.com/swlh/a-technical-guide-on-rnn-lstm-gru-for-stock-price-prediction-bce2f7f30346 prep ...
0
votes
0answers
25 views

How to Classify Game Stages Based on Bitrate Time Series Data

I need suggestions for my project and would be glad if you would give me a hand. I have a dataset of frames obtained from the old-school game DOOM. Each frame in the dataset has the following columns: ...
0
votes
1answer
78 views

How to improve LSTM accuracy on multiclass text classification?

So, I'm trying to build a LSTM model to classify multiclass text label. The goal is to make a prediction about user rating (1, 2, 3, 4, 5) based on their review. My hyperparameter is like this: ...
0
votes
0answers
62 views

Pytorch: how to pass the hidden state between the samples in LSTM?

I am trying to boost the performance of a object detection task with sequential information, using ConvLSTM. A typical ConvLSTM model takes a 5D tensor with shape ...
1
vote
0answers
21 views

Can Batch Normalization replace tanh in RNN?

Question Can Batch Normalization (BN) be inserted in RNN after $x_t@W_{xh}$, and after $h_{t-1}@W_{hh}$ to remove $f=tanh$ and bias $b_h$? If possible, will this ...
2
votes
0answers
115 views

Elman RNN with keras

I have to perform multi-step multivariate forecasting of time series, using keras. I found an example where LSTM is used. I could modify that example replacing LSTM with SimpleRNN. Now I would like to ...
1
vote
0answers
30 views

Why don't we set initial hidden states in RNN to random small numbers like we do to the weights?

I'm following a couple of tutorials on RNNs and the instructor said that we should always set the initial hidden state in our RNNs to a tensor of all zeros and I couldn't really understand why. Even ...
0
votes
0answers
14 views

speech emotion recongtion cnn or rnn?

I want to build a speech emotion classifier and I labeled my data into 3 emotions {negative, neutral, positive} the speech files I have are different of length, thus my audio features (mfcc,zcr, etc.) ...
0
votes
0answers
11 views

Pytorch RNN with no nonlinearity

Is it possible to implement an RNN layer with no nonlinearity in Pytorch like in Keras where one can set the activation to linear? By removing the nonlinearlity, I want to implement a first-order ...
1
vote
1answer
28 views

Do Recurrent Neural Networks assume stationarity or just a general kind of sequential dependence?

Just when I thought I had convinced myself that RNNs make no other assumption about a sequence other than that there are dependencies between the inputs and that (in the case of monodirectional RNNs) ...
0
votes
0answers
20 views

PyTorch LSTM with varying time steps

Is it possible to create an LSTM in PyTorch where the time steps are varying? For example, heights where measurements are taken at various times. The data might look like this: Person id Inches tall ...
0
votes
0answers
14 views

Treating Null Degrees of an Angle

I have a dataset that measures the flight details of objects, based on what action was performed. It looks similar to below: ...
1
vote
1answer
16 views

How can I convert my predictions to text after predicting using RNN?

I'm building PoS tagger for our language. I give tokens to the words and tags using Tokenizer(). Functions for word and tag are different. ...
1
vote
0answers
30 views

Padding before or after truncation for stateful LSTM in Keras

I am training an LSTM on a dataset with variable timesteps (between 10 and 6000). Using the truncated backpropagation through time (TBPTT) technique, I am truncating the sequences to windows of 128 ...
0
votes
0answers
9 views

RNN predict across new timeseries

For simplicity, I assume that the following timeseries $X_{A}$ and $Y_{A}$ are univariate. I am familiar with training RNN networks, such as LSTM, on a given timeseries $X_{A} = (X_{A0}, X_{A1}, X_{A2}...
-1
votes
2answers
23 views

Optimisation of neural networks

Do neural networks get optimized by trial and error, by data scientists, or is there some way of optimizing values through accurate mathematical equations?

1
2 3 4 5
14