Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [recurrent-neural-net]

The tag has no usage guidance.

0
votes
1answer
3 views

Understanding Exclusive-OR predictions in Elman network

I have been reading Elman network paper, which can be found Here. in page 185, under Exclusive-OR section it was written as follows. Notice that, given the temporal structure of this sequence, it ...
0
votes
0answers
11 views

Normalize between -1 and 1 or 0 and 1 (for LSTM)

Looking at various examples on the Internet I see some people normalize between -1 and 1, and others between 0 and 1. Is there any reason people choose one over the other? Assuming I'm using the ...
0
votes
0answers
18 views

Which time step output should be used in a LSTM network?

Let's take a LSTM network with one layer and two hidden units. Let's take that the number of time steps are 4, then the input x is: \begin{align} x = \big(x\small(t),\space x\small(t-1),\space x\...
0
votes
0answers
9 views

Using a Neural Network to predict the limes of a simple exponential load function

I am currently working on the following problem in Keras to get into Neural Networks. I do want to predict the limes of exponential load function load_e with two ...
0
votes
1answer
23 views

Bidirectional GRU: validation loss stuck on plateau diverges from well performing training loss

tl;dr: What's the interpretation of the validation loss decreasing faster than training loss at first but then get stuck on a plateau earlier and stop decreasing? The accuracy behaviour is similar. ...
2
votes
0answers
36 views

GRU learns small-scale features, but misses large scales

Playing around with weather data, I have setup a simple RNN with one layer of GRUs. It is trained to recover the temperature of the next day, given weather data of the last 5 days, each with 1 hour ...
2
votes
1answer
23 views

How many Hidden Layers and Neurons should I use in an RNN?

I am very new to neural networks and machine learning and I have been making a Bitcoin price predictor to learn it. I was wondering about the number of hidden layers I'd need in a recurrent neural net ...
1
vote
1answer
54 views

Stacking LSTM layers

Can someone please tell me the difference between those stacked LSTM layers? First image is given in this question and second image is given in this article. So far what I learned about stacking LSTM ...
1
vote
2answers
29 views

Why the RNN has input shape error?

My x_train shape is (798,3) and y_train input shape is (798, 1). I am creating a RNN like this ...
0
votes
1answer
13 views

practical improvements worth trying over plain LSTM in text classification?

I have a dataset of about 1 million tweets corresponding to about 30,000 user accounts, labelled with binary data (classifying the tweet as written by a bot). With that amount of data, I could use a ...
0
votes
0answers
21 views

Understanding LSTM/RNN structure

In keras when we apply LSTM/RNN model, we specify the node [i.e.,LSTM(128)]. I have a doubt how it actually works. From the LSTM/RNN unfolding image or description, I found that each RNN cell take one ...
0
votes
0answers
16 views

Neural Network stimuli propagation

Is not clear to me how exactly the stimuli proagate through a neural network. It's pretty clear how it should work in a feed-forward network but not in a more complex one. If i have understood, if ...
0
votes
1answer
25 views

Why does my LSTM perform better when randomizing training subset vs. standard batch training?

I am training a simple LSTM network using Keras to predict time series values. It is a simple 2-layer LSTM. I get the best performance when I train on subsets of the training set that start at random ...
0
votes
0answers
22 views

Backpropagation through time - How many Layers will an unfold produce?

In terms of Recurrent Neural Networks a backpropagation through time is used. That means, a RNN oder LSTM layer in Keras will be unfolded to x layers and backpropagation is performed on this unfolded ...
0
votes
0answers
11 views

idea for a Neural net for price prediction

i'm currently a day trader, with my own process for taking trades, and i'd like to build a neural net to replicate my thought process. I've coded in VBA plenty (don't laugh....its been useful) but ...
0
votes
1answer
51 views

Recurrent neural network (LSTM) dimensions error

I have data in a dataframe named ddf as follows: ...
0
votes
0answers
7 views

FAQ matching system based on RNN

Is it reasonable to apply interaction on CNN based embedding for long answers, with LSTM based embedding for short questions, in a QA system? I’m building a FAQ system that matches a query into most ...
1
vote
0answers
47 views

Shaping data for ConvLSTM for many-to-one image model

Ultimately, I am trying to obtain a binary segmentation mask for an image sequence. I have n number of image sequences, each with 500 greyscale images of size 256px by 400px. Each of these sequences ...
1
vote
0answers
16 views

Model Joint Probability of N Words Appearing Together in a Sentence

Assume that we have a large corpus of texts to train with. Given N words as input, I want to model the joint probability $p(x_1, x_2, ..., x_N)$ of these words appearing together in a sentence. More ...
3
votes
2answers
41 views

What does the one function $\mathbf{1}_{i,y^{(t)}}$ exactly mean in backward propagation of RNN in the book “Deep learning” of Bengio

It confused me for a long time what is $\mathbf{1}_{i,y^{(t)}}$ exactly mean in (10.18) below. It is in the Chapter 10 on RNN of the book LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep ...
1
vote
0answers
22 views

Addressing mechanisms in neural turing machine

In a Neural Turing Machine,why wasn't an absolute random access mechanism used?We are reading and writing based on the weighting emitted by the read/write head and this weighting is being generated by ...
4
votes
1answer
62 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
2
votes
1answer
87 views

Neural Network - distinguishing between several normalized values is impossible?

It's a common practice to normalize inputs to the neural Network. Let's assume we have a vector of activations. One of techniques, the Layer Normalization simply looks at the vector's components, ...
2
votes
0answers
21 views

Matrix multiplication issue (shapes not alligned)

I am building an RNN using numpy only and have started on the forward propagation section. However i am having some issues aligning my matrices. The issue is on this line: ...
1
vote
0answers
19 views

Simple explanation of LSTM data set and training phase

I cannot understand the training procedure of the LSTM (and other recurrent nets). My data is time series of length 2000 points. As suggested on the internet (and keras framework), this should be ...
2
votes
1answer
103 views

Why is predicted rainfall by LSTM coming negative for some data points?

I have used supervised learning with LSTM network using tanh activation function and 0.1 dropout for time series prediction.my loss='mean_squared_error', optimizer='adam'. The predicted time series is ...
1
vote
0answers
151 views

Multivariable time-series forecast with NN vs RNN

I am doing some comparison of RNN with other methodologies to check if the RNNs can improve some more "classical" models. In fact, I am doing it with multiple features (similar to what was done here) ...
1
vote
1answer
99 views

What's the difference of stateless LSTM and a normal feed-forward NN?

From what I understand, the whole point of LSTM is for the network to establish long-term dependencies in the data, i.e. an event happening now may be in some way determined by something that happened ...
0
votes
0answers
6 views

What is an example of an STDP general equation for a Spiking Neural Network?

I have been reading many articles on SNNs, and I understand the different instances of STDP such as locality, boundedness etc. But what is the general equation for such a model? For example ...
0
votes
1answer
73 views

TimeDistributed with different input / output sequence length

I'm looking into using TimeDistributed in my LSTM to see if it would improve the accuracy of my model. I'll be honest, I'm still not 100% sure what the specific ...
0
votes
1answer
23 views

What is the difference between these two methods of batching with stateful LSTM

With stateful LSTM, are these two methods effectively the same: ...
1
vote
0answers
98 views

For stateful LSTM, does sequence length matter?

With stateful LSTM the entire state is retained between both the sequences in the batch that is submitted, and even between separate batches until ...
1
vote
2answers
64 views

What are the benefits and tradeoffs of a 1D conv vs a multi-input seq2seq LSTM model?

I have 6 sequences, s1,..,s6. Using all sequences I want to predict a binary vector q = [0,0,0,1,1,1,0,0,0,1,1,1,...], which is a mask of the activity of the 6 sequences. I have looked at seq2seq ...
1
vote
1answer
26 views

Why does a filter need to be applied to the output of the input gate before cell state is added to?

In a neural network there are 4 gates: input, output, forget and a gate whose output performs element wise multiplication with the output of the input gate, which is added to the cell state (I don't ...
0
votes
0answers
15 views

Activity Detection

I am attempting to classify every point in a sound file as either being active or inactive. I have a binary mask of my training data which represents when they are active or not. Not every sound file ...
1
vote
0answers
17 views

If an NMT dataset is artificially enlarged by splitting sequences up, should it still train for the same number of epochs?

I have a neural machine translation dataset that looks something like this (letters represent words in an input sentence): ...
0
votes
1answer
47 views

Validation data for multi-series stateful LSTM

With stateful LSTM the network state is propagated to subsequent sequences and batches. I have multiple data files with data that I present to the network for training (making this multi-series). My ...
0
votes
1answer
328 views

Loss function for an RNN used for binary classification

I'm using an RNN consisting of GRU cells to compare two bounding box trajectories and determine whether they belong to the same agent or not. In other words, I am only interested in a single final ...
3
votes
1answer
135 views

Stateful LSTM : Using different training window

Would it make sense for stateful LSTM (or LSTM in general) if in one epoch I feed [0-9],[10-19],[20-29],[30-39]...[990-999] (with corresponding labels/Y data) from my dataset. When I've presented all ...
2
votes
1answer
239 views

Training with multi-series of different length with stateful LSTM

I'm training a stateful LSTM. My data is stored in a series of files, each file relates to a certain city. For each city I might have different amount of data, so City A I might have 4000 days, but ...
1
vote
1answer
64 views

Training a LSTM/any other deep learning model with temporal as well as non temporal attributes

I'm working on a project to predict the usage of all the files(rough frequency of usage) in a filesystem(a company server on which 100s of company employees are active) in near future(say the next 1 ...
0
votes
1answer
188 views

Types of Recurrent Neural Networks

I have a question about types of RNN. Ian Goodfellow in his book Deep Learning writes: Some examples of important design patterns for recurrent neural networks include the following: • ...
0
votes
2answers
322 views

Multivariate and multi-series LSTM

I am trying to create a pollution prediction LSTM. I've seen an example on the web to cater for a Multivariate LSTM to predict the pollution levels for one city (Beijing), but what about more than one ...
1
vote
1answer
42 views

How do I use rnn to forecast to n periods with limited data?

So this is my 1st time trying to run a small time-series dataset through an RNN, but after a lot of searching, I haven't been able to find, 1. How I can use this to forecast to n periods ? (like in ...
0
votes
0answers
167 views

LSTM with teacher forcing - NN fails to predict the sequence

I'm experimenting with LSTMs in Keras, and I'm trying to use the teacher forcing method in order to train a network to continue sine-waves (of randomly generated wavelengths, phase, and length). (...
1
vote
0answers
110 views

Input representation in a neural network [closed]

I have been going through this book (https://www.cs.toronto.edu/~graves/preprint.pdf) on sequence labelling by Alex Graves. On page 29, under the section of input representation he states that the ...
0
votes
1answer
91 views

Creating an LSTM NN for Fault Classification with Keras

Goal: Classify time series data from a wind turbine as being anomalous or non-anomalus in real time and from there predict what the anomaly is in a later, more refined, model. I have a CSV file with ...
4
votes
0answers
123 views

How are ANN's, RNN's related to logistic regression and CRF's?

This question is about placing the classes of neural networks in perspective to other models. In "An Introduction to Conditional Random Fields" by Sutton and McCallum, the following figure is ...
1
vote
0answers
74 views

Find most important inputs of LSTM-RNN for multivariate time series modeling

The question is similar to the one asked in SO, but I think it is more appropriate to be here. It remains anyway unanswered. Assuming multivariate time series, how to evaluate the importance of ...
2
votes
2answers
448 views

How to train the same RNN over multiple series?

I have multiple separate time series and would like to train the same LSTM network on them. How to do in this situation? I can't just concatenate timeseries (along time), because I am afraid network ...