Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

0
votes
0answers
15 views

very low val_acc in LSTM for predicting numbers in sequence

I have problem with my LSTM network or my data. I woudl like to create LSTM network to predict number based on previous N numbers. This numbers will be encoded piano notes. About my dataset: 1. I ...
1
vote
0answers
33 views

LSTM prediction for many multivariate time series

Let's say we have 8,000 different time series where each of them has 10,000 samples and 25 features. The goal is to have an LSTM sequence to sequence model (using Keras) where one can use a sequence ...
0
votes
0answers
9 views

Why are reservoir computer so useful for hardware implementations

I often read (e.g. here or in this question) that Reservoir Computer (RC) are useful in the field of Neuromorphic Computing where they can serve as efficient implementations of neural networks in ...
1
vote
0answers
24 views

Generate sentences using given data [closed]

I am working on an automated insights generation use case where I want to generate meaningful sentences from given aggregated data. For example, Data: ...
1
vote
2answers
42 views

Training LSTM with different sequence lengths in Keras functional api

I am trying to train an LSTM model using Keras functional API. My training data is of shape: >>> data.shape() (100000,variable_sequence_lengths,295) ...
0
votes
0answers
14 views

What does “large sparse” and “small dense” means?

In a paper I'm reading today it's written : For a fixed parameter count, we discover that large sparse WaveRNNs significantly outperform small dense WaveRNNs and that this relationship holds up ...
0
votes
0answers
8 views

Train LSTM RNN with multiple different sets of time series data in Keras

I am trying to set up a program where an airplane is taking off from one city and flying to another. Depending on a number of factors it can take different routes to get to the city. Since some ...
0
votes
0answers
11 views

What will go wrong if we apply linear or other types of regression to translate sentences between two languages?

Disclaimer: I asked the question at https://stats.stackexchange.com/questions/408463/what-will-go-wrong-if-we-apply-linear-or-other-types-of-regression-to-translate, but didn't get any response, so I'...
1
vote
1answer
25 views

How can I go about building a model for large number of outputs?

I have previously worked on small-scale feedforward neural network problems. But I have started working on a new project where the goal is to predict air quality in 25 locations throughout the ...
1
vote
0answers
53 views

High Training Accuracy, Poor Validation, Test Accuracy

I am a beginner exploring Deep learning. I am trying to train a classifier (9 classes) with images as the input to my CNN followed by Bidirectional LSTM architecture. My model rapidly achieves a ...
1
vote
1answer
32 views

Is it possible that a CNN has better accuracy than RNN in word classification?

So I found something strange once I compared the accuracy of the prediction of a class for a question between a CNN and an RNN (GRU). The CNN achieved 0.87 accuracy over the RNN (GRU) with 0.7520 ...
1
vote
0answers
17 views

Time Series Forecasting for Multiple Customers using one RNN

I have a product which has univariate and also multivariate time series data from multiple customers. I have variable amount of data available. Ranging between couple of years to couple of months. ...
3
votes
0answers
16 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
1
vote
0answers
12 views

Using TF Dataset API to process sequences for stateful RNN

I am trying to use the TensorFlow (v1.13) Dataset API to save and load long sequences for a stateful RNN. Basically, lets say I have n_seq sequences, each fixed ...
0
votes
0answers
23 views

Time series output of LSTM network has a much lower scale than the input scale

I'm trying to use an LSTM network to predict a sequence of a time series variable. I'm trying to predict a sequence of 3 elements based on the sequence of the previous 6 elements. The Keras code that ...
0
votes
0answers
13 views

Pytorch's pack_padded_sequence in Tensorflow?

If we do not use pack_padded_sequence of Pytorch, what will happen to the eval result? How to implement Pytorch's ...
1
vote
0answers
16 views

How Tensorflow text prediction predicts without softmax activation

In the Colab notebook here: RNN text generation in def generate_text(), there is ...
1
vote
0answers
36 views

How to apply an RNN to forecast non-stationary time series?

Is it possible to predict a time series which is non-stationary, in the sense that, the dependent variable Y have an increasing trend? Therefore, the highest value of $Y$ in the training set may be ...
0
votes
0answers
12 views

Hybrid Model : RNN + MLP + RNN

I am trying to develop a model, as follow: an RNN with three LSTM takes in the input (5|1|54), the 5 previous days and 54 feature. In the end of the first RNN I would like to take the mean and std of ...
1
vote
1answer
40 views

How to implement this CNN architecture in Keras

I am trying to implement in Keras the CNN architecture used by Rajpurkar et al and illustrated below: I am particularly confused about that max pool that is shown ...
2
votes
1answer
73 views

How to design batches in a stateful RNN

I am using TF Eager to train a stateful RNN (GRU). I have several variable length time sequences about 1 minute long which I split into windows of length 1s. In TF Eager, like in Keras, if ...
0
votes
1answer
13 views

Why don't we gradually update the activation parameters in RNN from one activation to the next as the network is learning more?

I'm very new to (unidirectional, vanilla) RNN and sequence modeling in general, and all I understood about the motivation on having the connection between two successive hidden layers/activation is ...
0
votes
0answers
22 views

Duplicate QUORA question detection:Kaggle Dataset

I have tried to use 2 BILSTMs along with the attention layer but the validation accuracy is not improving at all. Could anyone suggest an alternative to increase the accuracy? Layer structuring: <...
0
votes
0answers
26 views

Using an RNN to predict fantasy football results

So I have a couple questions about the design of a neural network. I'm trying to create a neural network to predict the number of fantasy points a player will score in a given week. First of all, I ...
0
votes
1answer
24 views

What is used for Machine Translation besides RNN

I am doing a university report and it seems that encode-decode RNN are optimal for Machine Translation. I need something else to compare it to but I can't seem to make a proper google search for it. ...
0
votes
1answer
21 views

What are the key differences between a MLP with lagged features and a RNN

I've been working with MLP's for a while. Whenever I assumed that the past values of a feature might be useful for predicting the future values of Y, I would just create a new column in my data frame ...
3
votes
1answer
33 views

Padding sequences for neural sequence models (RNNs)

I am padding sequences for a GRU based classifier that I am building in Keras. I'm wondering if there's any accepted best practice for padding the leading or trailing side of the sequence. E.g. <...
0
votes
1answer
42 views

Policy gradient/REINFORCE algorithm with RNN: why does this converge with SGM but not Adam?

I am working on training RNN model on caption generation with REINFORCE algorithm. I adopt self-critic strategy (see paper Self-critical Sequence Training for Image Captioning) to reduce the variance. ...
0
votes
0answers
9 views

Training error in RNN overshoots after considerable amount of iterations

I have trained a RNN model with pytorch. The training error overshoots after around 7.5k iterations. I have used gradient clipping and MultistepLR to decay the learning rate. Training Error on log ...
2
votes
1answer
42 views

Do I need to engineer lagged features when creating an LSTM for time series forecasting?

Long short-term memory networks are fairly complicated and I haven't completely wrapped my head around them. It seems to me like the big gain in LSTMs for time series forecasting is the lacking ...
0
votes
0answers
34 views

Issues with using stateful in StackedRNN cells

I am having issues with using the 'stateful' feature when building stacked RNN using LSTMCell object. I am following the instructions on tensorflow on how to set 'stateful = True' by passing the '...
0
votes
0answers
45 views

Seq2seq model that gets as input a sentence and outputs the same sentence

I tried to implement a model that takes as input sentences, which are hate_tweets and outputs exactly the same sentences. For this reason, I gave Input to the encoder and decoder exactly the same ...
0
votes
0answers
66 views

Is it always better to using stacked LSTM than single LSTM?

I am currently studying LSTM and RNNs. I came across several concepts like Multidimensional LSTM and Stacked LSTM. I have used Stacked LSTM and it gives me a better performance than single LSTM. As ...
1
vote
0answers
86 views

Zero padding for LSTM input

I am building a text-generation model. In the first layer, I am using Word2Vec embeddings. Now since the input is sentences they are variable length and I am padding them with zero. The input is ...
0
votes
1answer
17 views

Help required to implement the below model using Bi-GRU

As you can see in above images I need to model Bi-GRUs stacked as shown in table which takes input (N,1,64) and outputs (N,204). The input data is binary number stream and so is output data. Can ...
0
votes
0answers
14 views

Can I have multiple (a sequence of) predicted values as the output of a neural network model?

I have a multi-dimensional time series data, and I want to use these data to do a time-series prediction. That is, the target(ground truth) of the training data is in a time series format instead of ...
2
votes
2answers
56 views

understanding linear algebra of a forget gate

This blog covers the basics of LSTMs. A forget gate is defined as : $$f_t = \sigma(W_f \cdot [h_{t-1}, x_t]+ b_f)$$ At this point the linear algebra confuses me more than it should. The syntax of $...
3
votes
1answer
95 views

LSTM forecasting on multivariate time series

I'm new to RNNs and LSTM and would like some direction with a problem I have. I have a data set containing system metrics (like CPU utilization, disk operations, memory use) of an AWS EC2 instance ...
1
vote
1answer
34 views

Stuck on building a customer support chatbot from scratch using reddit dataset

I've a trained model who can mimic day to day conversation occurring on reddit. But, here my problem is that I want it to reply to a specific use cases based on the vocabulary it had learned. Summary:...
1
vote
1answer
23 views

Why do recurrent layers work better than simple feed-forward networks?

On a time series problem that we try to solve using RNNs, the input usually has the shape $input features \times timesteps \times batchsize$ and we then feed this ...
1
vote
0answers
31 views

Music Generation LSTM not learning (Keras)

I am trying to train a RNN in keras to produce music but I am having difficulty training it. The loss seems to remain fairly high and constant despite me changing the hidden size and number of layers ...
1
vote
2answers
129 views

extraction information from resume

I have a project in machine learning in which I need to analyze a curriculum vitae. for that I have to write a python program. It uses basic techniques of Natural Language Processing like word ...
1
vote
0answers
24 views

How to prepare data for time series RNN

Our goal is to predict different mixtures of gases in case of an air pollution or in space shuttles.We have measurements that are taken from medical devices and we are taking measurements from ...
1
vote
0answers
58 views

Should I remove the trend from timeseries when using DeepAR

I saw that for some other algorithms for timeseries data it is advised to remove trend and seasonality before doing the prediction (ex: ARIMA and LSTM) I figured out from the paper that SageMaker's ...
1
vote
1answer
21 views

Does CRNN use sparse tensor value for its label?

I just read paper about cnn + rnn for text recognition. The labels of dataset is tensor of char index (e.g [0, 1, 2 ] for image with label "abc"). Since the label of each input has different length ...
2
votes
1answer
56 views

Understanding LSTM structure

I am trying to learn LSTM's, and struggling a bit with the structure and the inputs/outputs of LSTM layers. Say I have a network definition like this: ...
2
votes
0answers
159 views

Train LSTM model with multiple time series

I am predicting energy usage for a bedroom within a school residential building with date, temperature and humidity as input features, using 7 time-steps and predicting for one-day (one-timestep). I ...
1
vote
0answers
20 views

Any special considerations on learning based on a cyclic timeseries using LSTM?

I have a time-serie over one year. For each 15 minutes in daylight hours, there is a data point. So basically, a sequence for each day. I would like to benefit this information if possible, I mean ...
4
votes
2answers
257 views

Understanding output of LSTM for regression

Please see the update, below. I am working with embeddings and wanted to see how feasible it is to predict some scores attached to some sequences of words. The details of the scores are not important....