Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

1
vote
0answers
23 views

Zero padding for LSTM input

I am building a text-generation model. In the first layer, I am using Word2Vec embeddings. Now since the input is sentences they are variable length and I am padding them with zero. The input is ...
0
votes
1answer
14 views

Help required to implement the below model using Bi-GRU

As you can see in above images I need to model Bi-GRUs stacked as shown in table which takes input (N,1,64) and outputs (N,204). The input data is binary number stream and so is output data. Can ...
0
votes
0answers
17 views

Loss function minimizing by pushing precision and recall to 0

I am designing a Recurrent Neural Network with LSTM cells (3, 5, 3 at the moment) to classify highly skewed data using the keras framework with Python. There are 8,640 time steps per day (10 second ...
0
votes
0answers
13 views

Can I have multiple (a sequence of) predicted values as the output of a neural network model?

I have a multi-dimensional time series data, and I want to use these data to do a time-series prediction. That is, the target(ground truth) of the training data is in a time series format instead of ...
2
votes
2answers
51 views

understanding linear algebra of a forget gate

This blog covers the basics of LSTMs. A forget gate is defined as : $$f_t = \sigma(W_f \cdot [h_{t-1}, x_t]+ b_f)$$ At this point the linear algebra confuses me more than it should. The syntax of $...
1
vote
1answer
73 views

LSTM forecasting on multivariate time series

I'm new to RNNs and LSTM and would like some direction with a problem I have. I have a data set containing system metrics (like CPU utilization, disk operations, memory use) of an AWS EC2 instance ...
1
vote
1answer
23 views

Stuck on building a customer support chatbot from scratch using reddit dataset

I've a trained model who can mimic day to day conversation occurring on reddit. But, here my problem is that I want it to reply to a specific use cases based on the vocabulary it had learned. Summary:...
0
votes
0answers
14 views

How can I train a RNN on different sets of data?

I have a practical question about RNN training, the answer to which I can't seem to find no matter how hard I try (googled a lot). Can I train an RNN that has a time element in it, where said time ...
1
vote
1answer
16 views

Why do recurrent layers work better than simple feed-forward networks?

On a time series problem that we try to solve using RNNs, the input usually has the shape $input features \times timesteps \times batchsize$ and we then feed this ...
0
votes
0answers
11 views

Small output range and delayed output? Predicting sine using LSTM

I have coded a very basic LSTM with forget gates (no libraries used). I'm trying to predict $0.5\cdot \sin(t + N)$ given $0.5\cdot\sin(t)$ as an exercise. I have tweaked the model, changing the ...
1
vote
0answers
17 views

Music Generation LSTM not learning (Keras)

I am trying to train a RNN in keras to produce music but I am having difficulty training it. The loss seems to remain fairly high and constant despite me changing the hidden size and number of layers ...
0
votes
0answers
12 views

Class weights for time-series data with imbalanced classes

I am having an issue where my loss is decreasing on every epoch, but my precision and recall are still extremely small. This is, in my understanding, because I have time-series of length 8639 and each ...
1
vote
2answers
37 views

extraction information from resume

I have a project in machine learning in which I need to analyze a curriculum vitae. for that I have to write a python program. It uses basic techniques of Natural Language Processing like word ...
1
vote
0answers
17 views

How to prepare data for time series RNN

Our goal is to predict different mixtures of gases in case of an air pollution or in space shuttles.We have measurements that are taken from medical devices and we are taking measurements from ...
0
votes
0answers
18 views

Should I remove the trend from timeseries when using DeepAR

I saw that for some other algorithms for timeseries data it is advised to remove trend and seasonality before doing the prediction (ex: ARIMA and LSTM) I figured out from the paper that SageMaker's ...
1
vote
1answer
17 views

Does CRNN use sparse tensor value for its label?

I just read paper about cnn + rnn for text recognition. The labels of dataset is tensor of char index (e.g [0, 1, 2 ] for image with label "abc"). Since the label of each input has different length ...
2
votes
1answer
38 views

Understanding LSTM structure

I am trying to learn LSTM's, and struggling a bit with the structure and the inputs/outputs of LSTM layers. Say I have a network definition like this: ...
1
vote
0answers
25 views

Train LSTM model with multiple time series

I am predicting energy usage for a bedroom within a school residential building with date, temperature and humidity as input features, using 7 time-steps and predicting for one-day (one-timestep). I ...
1
vote
0answers
18 views

Any special considerations on learning based on a cyclic timeseries using LSTM?

I have a time-serie over one year. For each 15 minutes in daylight hours, there is a data point. So basically, a sequence for each day. I would like to benefit this information if possible, I mean ...
4
votes
2answers
203 views

Understanding output of LSTM for regression

Please see the update, below. I am working with embeddings and wanted to see how feasible it is to predict some scores attached to some sequences of words. The details of the scores are not important....
0
votes
0answers
15 views

How to train LSTM with previous cell's prediction as an input in Keras?

At the moment I'm using a simple Keras model to learn a sequence of items and after it using the trained model to generate new sequences . I want to change the training to be in the same manner as ...
1
vote
0answers
25 views

preparing time series data for building a rnn

I am preparing time series data for to build an RNN model (LSTM). The data is collected from sensors installed in a mechanical plant. Consider I have data for input and output temperature of a ...
1
vote
1answer
37 views

How to put multiple features into RNN input vector

I am trying to code a recurrent neural network (LSTM) to create music in python and was considering using multiple features instead of just the note pitch as an input into the network. Initially I had ...
1
vote
0answers
9 views

Initialising states in a multilayer sequence to sequence model

With a sequence to sequence model where the enocoder and decoder are both comprised of one layer each, the initial state of the decoder is initialised to use the final states of the encoder layer. In ...
1
vote
0answers
64 views

Passing variable length sentences to Tensorflow LSTM

I have a tensorflow LSTM model for predicting the sentiment. I build the model with the maximum sequence length 150. (Maximum number of words) While making predictions, i have written the code as ...
0
votes
0answers
36 views

Replicating RNN within PyTorch

I tried to create a manual RNN and followed the official PyTorch example, which tries to classify a name to a language. I should note that it does indeed work. I'm not using the final logsoftmax, ...
0
votes
1answer
27 views

What is the best way to predict time series data? [closed]

I have monthly price data for tomatoes for the last 9 yrs for a particular town and I'm looking to predict the prices of tomatoes 6 months into the future. I had considered using Linear Regression in ...
0
votes
1answer
18 views

ML technique to predict next year output based on text quantities [closed]

I have a random data that I would like to predict how much a quantity will be in 2020. The data looks like this: ...
1
vote
1answer
41 views

Can RNN work with just one input?

We have a time series data.Is it possible to create a RNN such that there is only dimenson(or feature) in data(as shown in fig1).If yes,will it identify the pattern in time series data correctly? The ...
0
votes
1answer
32 views

Architecture help for multivariate input and output LSTM models

I am working on a sequence prediction problem using Keras. My end-goal is to have my input be several features, one being a categorical variable, that I have used 1 hot encoding for, and another ...
1
vote
1answer
39 views

Metrics for presenting RNN/LSTM result

I am working on a two different architecture based on LSTM model to predict the users next action based on the previous actions. I am wondering, what is the best way to present the result? Is it okay ...
0
votes
0answers
13 views

LSTM Autoencoder on Patterns of Labels

Currently, I am trying to do anomaly detection on univariate data consisting of labels. For example: [A, A, B, C] is good but [A, A, A, A] is anomalous. I'm dealing with more than just ABC. Is an LSTM ...
0
votes
1answer
45 views

Keras/TF: Making sure image training data shape is accurate for Time Distributed CNN+LSTM

The comprehensible data shape to me is like: (9186, 120, 120, 1) this means 9186 entry, of 120 by 120 pixel grey images. I learnt that using Time Distributed to design a CNN combined with an LSTM ...
0
votes
0answers
10 views

image caption generator

I see two models of image caption generator online: In the above model, the first LSTM cell of decoder takes the entire image as an input. In the above model, all the LSTM cells of the decoder take ...
0
votes
0answers
37 views

Best Architecture for LSTM Network for Stock Prediction

I am building an LSTM model to predict stock prices using TensorFlow. Is it best to structure the model so that it accepts $X=[x_0, x_1, ... x_{n-1}]$ and predicts $y=x_n$, or accepts $X=[x_0, x_1, ......
-1
votes
0answers
191 views

Modeling keras LSTM sequential vs functional api

I'm trying to compare 2 simple lstm's build with keras, one is of the Sequential api and the other is from the Functional api. Both models are getting sequences of 5 - each sequence has 5 features (e....
0
votes
1answer
80 views

How to pass 2 features to LSTM , one of them is one-hot-encoded with Keras?

I have a very simple LSTM model ...
-1
votes
0answers
37 views

Embedding when using an RNN and zero-padding on input strings

I have started developing an RNN/LSTM in tensorflow to take in short sentences (typically of length 5-15 tokens) along with a second categorical variable. The goal is to create an encoder-decoder to ...
-1
votes
0answers
17 views

Python LSTM Back propagation doesn't pass gradient check

I am trying to code a Recurrent neural network in python and I am having trouble getting the back propagation step to correctly calculate the gradients as when I check it using gradient checking the ...
0
votes
1answer
52 views

Architecture for linear regression with variable input where each input is n-sized one-hot encoded

I am relatively new to deep learning (got some experience with CNNs in PyTorch), and I am not sure how to tackle the following idea. I want to parse a sentence, e.g. I like trees., one-hot encoded the ...
1
vote
0answers
124 views

How to train the generator in a recurrent GAN (Keras)

I am trying to train a Recurrent GAN that is meant to generate geospatial movement data (sequences of 3-tuples of latitude, longitude and time). You may simply consider it a sequences of vectors with ...
2
votes
0answers
46 views

Build an Autocomplete model for document titles

I want to build an autocomplete model using RNN where input is article names (documents title). X: ['Billing', 'Loan status', 'Filling loan application', 'Contact Info', ...] The article name can ...
-1
votes
0answers
35 views

Neural Network architecture

I'm interested is it okay to use RNN encoder-decoder model for my task. I have train data with session_id, movie_id and ...
-1
votes
0answers
5 views

Sentence Embedding and mapping with the most likely sentence based on the inputted sentence

I have a text document and performed Sentence Embedding as Unsupervised learning in rnn. I want to map the most likely sentence in the document for the inputted Sentence.
1
vote
1answer
209 views

Recurrent Neural Network (LSTM) not converging during optimization

I am trying to train a RNN with text from wikipedia but I having having trouble getting the RNN to converge. I have tried increasing the batch size but it doesn't seem to be helping. All data is one ...
0
votes
1answer
27 views

Information about LTSM RNN backpropagation algorithm

I am attempting to make a LTSM RNN in python from scratch and I have completed the code for forward pass but I am struggling to find a clear outline of the equations I need to calculate to get the ...
-1
votes
0answers
49 views

time-series forecast with lstm and known future features

I am working on a lstm project. I want to use the model to predict a time-series and I am using a rolling window approach. I want to predict electricity consumption for industrial plants on a daily ...
1
vote
1answer
88 views

Need to make an multivariate RNN, confused about input shape?

So I've seen this: Keras LSTM with 1D time series And this: Multi-dimentional and multivariate Time-Series forecast (RNN/LSTM) Keras But I still don't quite get it. I have many, many, many accountIDs,...
2
votes
0answers
143 views

Recommended model for univariate or multivariate multistep ahead time series forecasting

I have a dataset consisting of recurring and non-recurring expense transactions from bank accounts, as well as other features describing the bank account and each transation. I aggregate these ...