Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

0
votes
1answer
14 views

Can RNN learn for each `t` in time from a whole new dataset (many entries)

Basically, my data set is not as simple multi-variate time-serie as it's often (to some extent) the case. For each month, I have N entries (not less than 3000). Can RNN of any variant (Please bear my ...
0
votes
1answer
3 views

Understanding Exclusive-OR predictions in Elman network

I have been reading Elman network paper, which can be found Here. in page 185, under Exclusive-OR section it was written as follows. Notice that, given the temporal structure of this sequence, it ...
0
votes
1answer
20 views

Outputs of an LSTM Cell

from each cell of lstm, what are the output's and what does they signify? i understand that there will be three outputs. A long term memory, short term memory and a output. But, i am little confused ...
0
votes
0answers
13 views

Using SMAPE as a loss function for an LSTM

I am currently working on a time series forecasting problem and am looking into using an LSTM. My final accuracy metric that I use to determine whether or not the forecast is good or not is defined ...
1
vote
0answers
9 views

How to predict multiple iterations in the future based on current close prices? Keras Python

I have data from CoinMarketCap for the close prices of Bitcoin. How can I use this data to predict multiple days into the future using past days with a function? I want the function to look like: <...
1
vote
0answers
12 views

LSTM/RNN seems to be failing at testing

I'm relatively new to ML, keras and tensorflow and I working with a dataset (kerastest.csv) that is 400 lines of this ...
0
votes
0answers
27 views

First Neural Network: Poor Quality of Predictions but low val_loss

I am a total newbie to ML, please be gentle :) I've created a RNN that should learn how to count. Input is a sequence of five consecutive numbers N, N+1, ..., N + 4...
0
votes
0answers
15 views

Can an LSTM learn correlations between time series and produce skillful predictions for individual time series?

I am trying to build a model that is capable of producing a multi-step forecast for many different time series. To keep the example simple, let's say I have three different time series, $T_1$, $T_2$ ...
0
votes
1answer
8 views

not quite sure about the difference between RNN and feed forward neural net

I'm a bit confused after reading this paper: https://arxiv.org/abs/1705.09851 on page 22, the author writes response: \begin{equation} Y = softmax(Z^{L-1}) \end{equation} and hidden state \begin{...
0
votes
0answers
16 views

Multiple entity extraction with character level RNN

I'm training a neural network to extract a certain kind of entities in a sentence (e.g. company names in a news title). Since I'm handling a multi-language corpus (especially CJK), which could be very ...
0
votes
0answers
21 views

Give Variable Length input to LSTM

My input data consist of list of list. Both list have dynamic length for every example like below. ...
-1
votes
0answers
10 views

Defining batch_size in the model.add vs model.fit

What is the difference between ...
0
votes
1answer
24 views

input_dim for Dense Layer after LSTM layers Keras

Do I need to specify the input_dim (which means the number of features in one row/sample) after adding the first LSTM layer for the later Dense layers? I was trying to create an architecture with 2 ...
0
votes
0answers
7 views

How can GPU speed up RNN with Sequence Data (no shuffle)?

It has been stated that CuDNNLSTM with GPU support greatly speeds up training compared to LSTM on CPU in Keras. But if I am working with sequence data in a time series, how can the data be processed ...
1
vote
1answer
18 views

No accuracy in Keras RNN Model with Bitcoin Data

I am very new to machine-learning and have made an RNN-LSTM model with no accuracy. My data has been normalized with MinMaxScaler from Sklearn and has a shape of has an input of shape (3, 2)... My ...
0
votes
0answers
14 views

How to prepare data for time series RNN

I want to use RNN on my data. The data is from a number of medical devices, it is a time series. My problem is that there are few types of files, each having data from a deferent source: File #1: ...
2
votes
1answer
42 views

TF-IDF Features vs Embedding Layer

Have you guys tried to compare the performance of TF-IDF features* with a shallow neural network classifier vs a deep neural network models like an RNN that has an embedding layer with word embedding ...
0
votes
1answer
23 views

Bidirectional GRU: validation loss stuck on plateau diverges from well performing training loss

tl;dr: What's the interpretation of the validation loss decreasing faster than training loss at first but then get stuck on a plateau earlier and stop decreasing? The accuracy behaviour is similar. ...
2
votes
1answer
23 views

How many Hidden Layers and Neurons should I use in an RNN?

I am very new to neural networks and machine learning and I have been making a Bitcoin price predictor to learn it. I was wondering about the number of hidden layers I'd need in a recurrent neural net ...
-1
votes
0answers
10 views

Training a RNN, Sequence to Sequence VS Sequence to One?

I would like my RNN to be able to be able to predict wether the person is likely to make a purchase based on website visits. Let's assume that I have data for a user in the following format: ...
0
votes
0answers
10 views

Using character level model to notes generation

I would like to use character based model for music generation. Predict note after N notes. (like character model based text generators). Do to this I would like to use RNN with LSTM in Keras. I ...
2
votes
2answers
21 views

What is the relation between input into LSTM and number of cells?

I want to train an LSTM network for time-series predictions, and want to get to the bottom of LSTM's. In my understanding, the number of cells in a single LSTM layer can vary. However, since each cell ...
0
votes
0answers
13 views

Sentence classification with tensorflow (Dynamic rnn)

I'm trying to implement a rnn model for sentence classification. Precisely this is the schema I'm trying to implement Sentences are first encoded with word2vec vectors and then fed into the rnn, they ...
0
votes
0answers
21 views

Understanding LSTM/RNN structure

In keras when we apply LSTM/RNN model, we specify the node [i.e.,LSTM(128)]. I have a doubt how it actually works. From the LSTM/RNN unfolding image or description, I found that each RNN cell take one ...
0
votes
0answers
11 views

How to feed huge datasets into Keras Models at a super fast rate?

I'm trying to find an efficient way to train a NN on a huge dataset. To understand the volume of the problem, I have 100x400mb csv files, which take up 20+GB in memory when loaded in pandas. Now the ...
0
votes
0answers
6 views

Truncating RNN due to limited resources

I am currently working with time series data and am able to fit it to a RNN in Keras. However, I am continually getting new data and it becomes computationally more expensive to train on all of the ...
0
votes
0answers
7 views

How to denormalize data used to train a RNN?

I am running the code defined in the "usage" section of this repo. I get the same output : Before training and testing, the data is normalized with the following code : ...
0
votes
0answers
22 views

Backpropagation through time - How many Layers will an unfold produce?

In terms of Recurrent Neural Networks a backpropagation through time is used. That means, a RNN oder LSTM layer in Keras will be unfolded to x layers and backpropagation is performed on this unfolded ...
0
votes
0answers
49 views

Classification of text using RNN LSTM

I havea problem statement of “Classification of text”. I am very new to machine learning and neural networks. So far, I have found this great example using pytorch. This implements basic RNN for ...
0
votes
0answers
13 views

Cause of validation accuracy and loss decreasing in tandem

I am training a 3-layer RNN over 50 epochs. I have created it using 256 cells and .3 dropout on each layer. The learning rate is 0.0001. The issue I am facing is that the validation accuracy is ...
0
votes
1answer
51 views

Recurrent neural network (LSTM) dimensions error

I have data in a dataframe named ddf as follows: ...
2
votes
1answer
15 views

Relation between amount of training samples and model depth?

When I add more hidden layers to my CNN (e.g. Dense Layers) it seems that the model needs more training samples to produce good results for classes with few training samples. In the single layer case ...
1
vote
1answer
27 views

Where is the output in the LSTM?

I'm trying to understand where the output of the LSTM is. Please refer to the following picture: http://colah.github.io/posts/2015-08-Understanding-LSTMs/ It seems that at each tilmestep, we output ...
0
votes
0answers
35 views

How to improve the under-fitting issue of my RNN Autoencoder on random squences?

I am trying to use RNN (seq-to-seq) as Autoencoder to reconstruct random sequences (classification). However, the model is suffering from under-fitting issue in which it can't improve the training ...
2
votes
1answer
62 views

Python - Predicting data based on multidimensional array with Keras

I've a list of data which is so called 3D array. Each of 10350 rows contains a 2D matrix with size of 150x16 (elements are float) (x_train). Corresponding training data for this huge array a linear ...
1
vote
0answers
24 views

Reinforcement learning - How to deal with varying number of actions which do number approximation

I am a new to Reinforcement learning, but I am trying to use RL in this task: Given a function definition in written e.g. in C with 1 to 10s of input arguments (only numerical ones - integer, float, ...
0
votes
0answers
18 views

Architecting an Attention network

Given the following data: ...
3
votes
2answers
41 views

What does the one function $\mathbf{1}_{i,y^{(t)}}$ exactly mean in backward propagation of RNN in the book “Deep learning” of Bengio

It confused me for a long time what is $\mathbf{1}_{i,y^{(t)}}$ exactly mean in (10.18) below. It is in the Chapter 10 on RNN of the book LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep ...
2
votes
2answers
33 views

Multiple-output vs single-output NNs

I'm trying to build a 5 input-5 output model using LSTM, where all the outputs are the same features as the inputs, predicted in the future. My question is: is it better to build 5 models, each with ...
1
vote
1answer
48 views

Should reinforcement learning always assume (PO)MDP?

I recently just started learning reinforcement learning and learned that reinforcement learning algorithms work under the assumption of MDP or POMDP. However as I read A3C and recent vision based deep ...
2
votes
1answer
20 views

Predicting t+1 from a set of sequences

Say I have have an experiment where I release a single rat into a maze and wait for it to reach the end. Say I also track this rat's position in the maze at various times. Let's do this $n$ times. Now,...
3
votes
0answers
31 views

RNN-based Predictions of Sine Waves with Frequency Different From Training Data

I am wondering if I can generate a sine wave with a frequency different from training data using RNN. For example, Using two training data of two time series, say 0[sec] ~ 10[sec] each: sin(t) and ...
6
votes
2answers
207 views

Dropout on which layers of LSTM?

Using a multi-layer LSTM with dropout, is it advisable to put dropout on all hidden layers as well as the output Dense layers? In Hinton's paper (which proposed ...
0
votes
0answers
31 views

How can I set initial state of LSTM in dynamic RNN for every single input in TensorFlow?

I am working on task where I have a state vector for every input example as opposed to setting state for every batch. is it possible to set the states of LSTM for every single input using Tensorflow? ...
2
votes
1answer
45 views

Are word embeddings further updated during training for document classification?

I am relatively new to the area of using word embeddings in NLP tasks. From a large corpus of documents, I train word2vec word embedding vectors and afterwards I am going to use these for document ...
1
vote
1answer
19 views

Understanding Embeddings input and output sizes

I have been trying for a while to understand the dimensionality of embeddings in neural networks and I think that finally things have clicked on my brain, however I would love to check whether or not ...
0
votes
0answers
17 views

Change training data for use in CNN -> RNN -> CTC model

What changes to do I need to make in order to utilize a CTC loss? If my training data is of the form: [({x1,x2,x3,x4,x5},{y1,y2,y3}), ({x1,x2,x3},{y1,y2}), ...] What changes to I need to make for the ...
0
votes
0answers
27 views

Learning a non-linear mapping using LSTM units: encountering overfitting

I am trying to use a recurrent neural network to perform sensor fusion for an inertial measurement unit. IMUs are commonly used in conjunction with a Kalman filter (KF), which performs both fusion of ...
1
vote
0answers
56 views

Sequence Embedding

Is there a way to get embedding for an ordered sequence of vectors? I want to get embeddings to feed them further into net i.e. train it to arbitrary loss function simultaneously for embeddings and ...
1
vote
0answers
74 views

Multivariate, multistep forecasting with LSTM

I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" ...