Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [recurrent-neural-net]

The tag has no usage guidance.

0
votes
0answers
23 views

Best Architecture for LSTM Network for Stock Prediction

I am building an LSTM model to predict stock prices using TensorFlow. Is it best to structure the model so that it accepts $X=[x_0, x_1, ... x_{n-1}]$ and predicts $y=x_n$, or accepts $X=[x_0, x_1, ......
0
votes
0answers
10 views

RNN for prediciting Development over time

Hey so I have biomarker measurements of individuals on many time points. Also at some points, the individuals get multiple injections irregulary (which are also recorded) which impact the following ...
0
votes
1answer
43 views

Architecture for linear regression with variable input where each input is n-sized one-hot encoded

I am relatively new to deep learning (got some experience with CNNs in PyTorch), and I am not sure how to tackle the following idea. I want to parse a sentence, e.g. I like trees., one-hot encoded the ...
1
vote
0answers
11 views

Mini-batches with sequential data

I am a little bit confused. When using mini-batches, it is a good idea to shuffle. This will not work if the training examples are dependent on each other, e.g. 5 minute voltage measurement data, ...
0
votes
0answers
17 views

Backpropagation through LSTM and MLP layers

For didactic reason, I am currently implementing in numpy an LSTM network for classifications. I need to add on top of the LSTM another fully connected layer, because I don't want the output to have ...
0
votes
0answers
10 views

Multivariate LSTM RMSE value is getting very high

I want to predict a time series with multiple variables. I am using Keras's LSTM class. Here is my data set description : I want to predict var1(t-1) and my X variables are var3(t-1) , var4(t-1) , ...
0
votes
0answers
33 views

Neural Network architecture

I'm interested is it okay to use RNN encoder-decoder model for my task. I have train data with session_id, movie_id and ...
2
votes
2answers
42 views

Validation loss is not decreasing

I am trying to train a LSTM model. Here is train and validation loss graph. Is this model suffering from overfitting problem ?
1
vote
1answer
29 views

Need to make an multivariate RNN, confused about input shape?

So I've seen this: Keras LSTM with 1D time series And this: Multi-dimentional and multivariate Time-Series forecast (RNN/LSTM) Keras But I still don't quite get it. I have many, many, many accountIDs,...
2
votes
1answer
38 views

Sequential Modelling: Multiple Sequence to One or Sequence to Sequence

Suppose I have a single sequence of $x_1, x_2, ..., x_n$ and corresponding labels $y_1, y_2, ..., y_n$. An example would be a person makes website visits $x_i$ and the label $y_i$ tells us if there ...
0
votes
1answer
31 views

What is the advantage of using RNN with fixed timestep length over Neural Network?

More often than not, I see RNNs being used with fixed length timesteps. So what is the difference between the following two networks? RNN with timestep length of 3 over sequence Xt. NN with inputs x(...
1
vote
1answer
15 views

Where can I download the toy benchmark dataset for RNNs?

I have read the paper: Simple Way to Initialize Recurrent Networks of Rectified Linear Units Where can I download the toy benchmark dataset for RNNs this paper mentions? I need addition problem ...
-1
votes
0answers
33 views

Questions about Backpropagation Through Time for Gated Recurrent Unit?

I'm trying to implement it myself so I can understand it more. I ended up deriving the gradients myself. So my understanding is that if $t=T$ is the terminal time index, and suppose you have forward ...
0
votes
1answer
13 views

Recurrent Neural Networks Over Multiple Documents Over Time

So in my head, I have an idea about what this architecture should look like, or at least behave, but I am having trouble implementing it. So let me describe the problem, and if anyone has an idea on ...
0
votes
0answers
20 views

LSTM regressor seems to clip predictions

I have built a LSTM regressor with an average absolute error of 8%, what I find not bad since it is the first model... However, looking at the predictions, the network seems to be clipping them, I'm ...
0
votes
1answer
5 views

Understanding Exclusive-OR predictions in Elman network

I have been reading Elman network paper, which can be found Here. in page 185, under Exclusive-OR section it was written as follows. Notice that, given the temporal structure of this sequence, it ...
0
votes
0answers
17 views

Normalize between -1 and 1 or 0 and 1 (for LSTM)

Looking at various examples on the Internet I see some people normalize between -1 and 1, and others between 0 and 1. Is there any reason people choose one over the other? Assuming I'm using the ...
0
votes
0answers
29 views

Which time step output should be used in a LSTM network?

Let's take a LSTM network with one layer and two hidden units. Let's take that the number of time steps are 4, then the input x is: \begin{align} x = \big(x\small(t),\space x\small(t-1),\space x\...
0
votes
1answer
61 views

Bidirectional GRU: validation loss stuck on plateau diverges from well performing training loss

tl;dr: What's the interpretation of the validation loss decreasing faster than training loss at first but then get stuck on a plateau earlier and stop decreasing? The accuracy behaviour is similar. ...
2
votes
0answers
37 views

GRU learns small-scale features, but misses large scales

Playing around with weather data, I have setup a simple RNN with one layer of GRUs. It is trained to recover the temperature of the next day, given weather data of the last 5 days, each with 1 hour ...
2
votes
1answer
28 views

How many Hidden Layers and Neurons should I use in an RNN?

I am very new to neural networks and machine learning and I have been making a Bitcoin price predictor to learn it. I was wondering about the number of hidden layers I'd need in a recurrent neural net ...
1
vote
1answer
178 views

Stacking LSTM layers

Can someone please tell me the difference between those stacked LSTM layers? First image is given in this question and second image is given in this article. So far what I learned about stacking LSTM ...
1
vote
2answers
45 views

Why the RNN has input shape error?

My x_train shape is (798,3) and y_train input shape is (798, 1). I am creating a RNN like this ...
0
votes
2answers
32 views

practical improvements worth trying over plain LSTM in text classification?

I have a dataset of about 1 million tweets corresponding to about 30,000 user accounts, labelled with binary data (classifying the tweet as written by a bot). With that amount of data, I could use a ...
-1
votes
0answers
24 views

Understanding LSTM/RNN structure

In keras when we apply LSTM/RNN model, we specify the node [i.e.,LSTM(128)]. I have a doubt how it actually works. From the LSTM/RNN unfolding image or description, I found that each RNN cell take one ...
0
votes
0answers
17 views

Neural Network stimuli propagation

Is not clear to me how exactly the stimuli proagate through a neural network. It's pretty clear how it should work in a feed-forward network but not in a more complex one. If i have understood, if ...
0
votes
1answer
38 views

Why does my LSTM perform better when randomizing training subset vs. standard batch training?

I am training a simple LSTM network using Keras to predict time series values. It is a simple 2-layer LSTM. I get the best performance when I train on subsets of the training set that start at random ...
0
votes
0answers
43 views

Backpropagation through time - How many Layers will an unfold produce?

In terms of Recurrent Neural Networks a backpropagation through time is used. That means, a RNN oder LSTM layer in Keras will be unfolded to x layers and backpropagation is performed on this unfolded ...
-1
votes
0answers
12 views

idea for a Neural net for price prediction

i'm currently a day trader, with my own process for taking trades, and i'd like to build a neural net to replicate my thought process. I've coded in VBA plenty (don't laugh....its been useful) but ...
0
votes
1answer
87 views

Recurrent neural network (LSTM) dimensions error

I have data in a dataframe named ddf as follows: ...
1
vote
1answer
139 views

Shaping data for ConvLSTM for many-to-one image model

Ultimately, I am trying to obtain a binary segmentation mask for an image sequence. I have n number of image sequences, each with 500 greyscale images of size 256px by 400px. Each of these sequences ...
1
vote
0answers
19 views

Model Joint Probability of N Words Appearing Together in a Sentence

Assume that we have a large corpus of texts to train with. Given N words as input, I want to model the joint probability $p(x_1, x_2, ..., x_N)$ of these words appearing together in a sentence. More ...
3
votes
2answers
43 views

What does the one function $\mathbf{1}_{i,y^{(t)}}$ exactly mean in backward propagation of RNN in the book “Deep learning” of Bengio

It confused me for a long time what is $\mathbf{1}_{i,y^{(t)}}$ exactly mean in (10.18) below. It is in the Chapter 10 on RNN of the book LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep ...
1
vote
0answers
22 views

Addressing mechanisms in neural turing machine

In a Neural Turing Machine,why wasn't an absolute random access mechanism used?We are reading and writing based on the weighting emitted by the read/write head and this weighting is being generated by ...
4
votes
1answer
86 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
2
votes
1answer
109 views

Neural Network - distinguishing between several normalized values is impossible?

It's a common practice to normalize inputs to the neural Network. Let's assume we have a vector of activations. One of techniques, the Layer Normalization simply looks at the vector's components, ...
2
votes
0answers
47 views

Matrix multiplication issue (shapes not alligned)

I am building an RNN using numpy only and have started on the forward propagation section. However i am having some issues aligning my matrices. The issue is on this line: ...
1
vote
0answers
26 views

Simple explanation of LSTM data set and training phase

I cannot understand the training procedure of the LSTM (and other recurrent nets). My data is time series of length 2000 points. As suggested on the internet (and keras framework), this should be ...
2
votes
1answer
161 views

Why is predicted rainfall by LSTM coming negative for some data points?

I have used supervised learning with LSTM network using tanh activation function and 0.1 dropout for time series prediction.my loss='mean_squared_error', optimizer='adam'. The predicted time series is ...
1
vote
0answers
237 views

Multivariable time-series forecast with NN vs RNN

I am doing some comparison of RNN with other methodologies to check if the RNNs can improve some more "classical" models. In fact, I am doing it with multiple features (similar to what was done here) ...
2
votes
1answer
318 views

What's the difference of stateless LSTM and a normal feed-forward NN?

From what I understand, the whole point of LSTM is for the network to establish long-term dependencies in the data, i.e. an event happening now may be in some way determined by something that happened ...
0
votes
1answer
148 views

TimeDistributed with different input / output sequence length

I'm looking into using TimeDistributed in my LSTM to see if it would improve the accuracy of my model. I'll be honest, I'm still not 100% sure what the specific ...
0
votes
1answer
23 views

What is the difference between these two methods of batching with stateful LSTM

With stateful LSTM, are these two methods effectively the same: ...
1
vote
0answers
135 views

For stateful LSTM, does sequence length matter?

With stateful LSTM the entire state is retained between both the sequences in the batch that is submitted, and even between separate batches until ...
1
vote
2answers
88 views

What are the benefits and tradeoffs of a 1D conv vs a multi-input seq2seq LSTM model?

I have 6 sequences, s1,..,s6. Using all sequences I want to predict a binary vector q = [0,0,0,1,1,1,0,0,0,1,1,1,...], which is a mask of the activity of the 6 sequences. I have looked at seq2seq ...
1
vote
1answer
27 views

Why does a filter need to be applied to the output of the input gate before cell state is added to?

In a neural network there are 4 gates: input, output, forget and a gate whose output performs element wise multiplication with the output of the input gate, which is added to the cell state (I don't ...
0
votes
0answers
15 views

Activity Detection

I am attempting to classify every point in a sound file as either being active or inactive. I have a binary mask of my training data which represents when they are active or not. Not every sound file ...
1
vote
0answers
18 views

If an NMT dataset is artificially enlarged by splitting sequences up, should it still train for the same number of epochs?

I have a neural machine translation dataset that looks something like this (letters represent words in an input sentence): ...
0
votes
1answer
78 views

Validation data for multi-series stateful LSTM

With stateful LSTM the network state is propagated to subsequent sequences and batches. I have multiple data files with data that I present to the network for training (making this multi-series). My ...
0
votes
1answer
562 views

Loss function for an RNN used for binary classification

I'm using an RNN consisting of GRU cells to compare two bounding box trajectories and determine whether they belong to the same agent or not. In other words, I am only interested in a single final ...