Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [recurrent-neural-net]

The tag has no usage guidance.

0
votes
0answers
12 views

How to manipulate recurrent CNN model on sentence classification?

I learned how to build recurrent cnn model for text classification and sketched out my initial implementation. However, I am wondering how to transform recurrent <...
2
votes
2answers
123 views

Unnormalized Log Probability - RNN

I am going through the deep learning book by Goodfellow. In the RNN section I am stuck with the following: RNN is defined like following: And the equations are : Now the $O^{(t)}$ above is ...
1
vote
1answer
29 views

How to estimate the not available observation in time series data?

Suppose, I have a 30 seconds time-step observations of sports data, in some of the intervals the game was partially/fully stopped. I'm trying to prep the data for a time series analysis. Is it ...
1
vote
1answer
21 views

How to create a language translator from scratch?

I want to create a translator which can translate English, Korean and Tamil sentences into English sentence, I tried googletrans but is there any way to create something better than that using DL and ...
1
vote
0answers
15 views

Reinforcement learning - generating a matrix of continuous values with varying size for test data generation

Currently, I am using RL A3C algorithm for test data generation, where for a set of 30 functions written in C (mostly basic algorithms like Prime number checks, triangle validity, etc.) I try to ...
0
votes
1answer
20 views

Accuracy and Loss in MLP

I am trying to explore models for predicting whether the a team will win or lose based on features about the team and their opponent. My training data is 15k samples with 760 numerical features. Each ...
0
votes
1answer
30 views

principles of time series analysis by neural network models

I can understand for speech signals, words are correlated and therefore one should have a reason to believe that recurring NNs or LSTMs could predict by running some complex algorithm with weights and ...
0
votes
0answers
20 views

Is this code correct for a sequential model for time series pattern prediction Keras

posting this to stack exchange DS as I have also seen people answering keras related questions here! I have a question about pre-processing data in order to enter it into a sequential model in keras ...
0
votes
0answers
19 views

How to training the recurrent recommender system with LSTM?

Recently, I read a paper about recurrent recommender system, I am very curious about how it training its network. Assume I have the Netflix dataset as ...
0
votes
1answer
26 views

Is the LSTM share the same model parameter in each block?

I learn the LSTM recently, and a little bit confuse about the model parameters about LSTM, The follow is the LSTM structure And it is equation as (I slightly ignore the bias in each equation): $$...
0
votes
1answer
40 views

ML technique to predict next performance anomaly

I would like a direction for a ML technique to predict when an anomaly will occur in a system, like when the CPU use differs from a normal state. My data consists of system metrics collected from AWS ...
1
vote
2answers
61 views
1
vote
1answer
55 views

How to reshape data for LSTM training in multivariate sequence prediction

I want to build an LSTM model for customer behaviour. It's the first time for me working on a timeseries, so some concepts are not clear to me at all. My prediction problem is multidimensional, ...
0
votes
0answers
14 views

Neural Network Architecture for batch of time series data

Let's say I have a data set which is a 2-Dimensional Matrix as the input and I want to predict either 0 or 1 with regard to the entire 2-D matrix. Now each row in the 2-D matrix is a time series, i.e....
3
votes
0answers
42 views

LSTM Long Term Dependencies Keras

I am familiar with the LSTM unit (memory cell, forget gate, output gate etc) however I am struggling to see how this links to the LSTM implementation in Keras. In Keras the input data structure for X ...
1
vote
0answers
15 views

What should the size of the decoder output be in a sequence to sequence model

In a sequence to sequence model, a lot of the tutorials I have read state that the decoder target length should be the same as the encoder input length (https://blog.keras.io/building-autoencoders-in-...
1
vote
2answers
36 views

Is there a disadvantage to letting a model train for a large number of epochs?

I created a model to solve a time series forecasting problem. I had a limited amount of time series with which I could train the model therefore I decided to augment the data. The data augmentation ...
1
vote
0answers
239 views

How to determine feature importance in a neural network?

I have a neural network to solve a time series forecasting problem. It is a sequence-to-sequence neural network and currently it is trained on samples each with ten features. The performance of the ...
3
votes
1answer
63 views

Is an Arma model equivalent to a 1-layer Recurrent Neural Network without activation function?

Given a time series $f(t)$ to forecast, let us consider an Arma model of the form: $$ f(t) = c + \sum_{i=1}^p a_i f(t-i) + e(t) + \sum_{j=1}^q b_j e(t-j) $$ where $e(t)$ are the forecast errors. On ...
0
votes
0answers
35 views

Best Architecture for LSTM Network for Stock Prediction

I am building an LSTM model to predict stock prices using TensorFlow. Is it best to structure the model so that it accepts $X=[x_0, x_1, ... x_{n-1}]$ and predicts $y=x_n$, or accepts $X=[x_0, x_1, ......
0
votes
0answers
11 views

RNN for prediciting Development over time

Hey so I have biomarker measurements of individuals on many time points. Also at some points, the individuals get multiple injections irregulary (which are also recorded) which impact the following ...
0
votes
1answer
52 views

Architecture for linear regression with variable input where each input is n-sized one-hot encoded

I am relatively new to deep learning (got some experience with CNNs in PyTorch), and I am not sure how to tackle the following idea. I want to parse a sentence, e.g. I like trees., one-hot encoded the ...
1
vote
0answers
19 views

Mini-batches with sequential data

I am a little bit confused. When using mini-batches, it is a good idea to shuffle. This will not work if the training examples are dependent on each other, e.g. 5 minute voltage measurement data, ...
0
votes
0answers
34 views

Backpropagation through LSTM and MLP layers

For didactic reason, I am currently implementing in numpy an LSTM network for classifications. I need to add on top of the LSTM another fully connected layer, because I don't want the output to have ...
0
votes
0answers
46 views

Multivariate LSTM RMSE value is getting very high

I want to predict a time series with multiple variables. I am using Keras's LSTM class. Here is my data set description : I want to predict var1(t-1) and my X variables are var3(t-1) , var4(t-1) , ...
-1
votes
0answers
35 views

Neural Network architecture

I'm interested is it okay to use RNN encoder-decoder model for my task. I have train data with session_id, movie_id and ...
3
votes
2answers
235 views

Validation loss is not decreasing

I am trying to train a LSTM model. Here is train and validation loss graph. Is this model suffering from overfitting problem ?
1
vote
1answer
83 views

Need to make an multivariate RNN, confused about input shape?

So I've seen this: Keras LSTM with 1D time series And this: Multi-dimentional and multivariate Time-Series forecast (RNN/LSTM) Keras But I still don't quite get it. I have many, many, many accountIDs,...
2
votes
1answer
43 views

Sequential Modelling: Multiple Sequence to One or Sequence to Sequence

Suppose I have a single sequence of $x_1, x_2, ..., x_n$ and corresponding labels $y_1, y_2, ..., y_n$. An example would be a person makes website visits $x_i$ and the label $y_i$ tells us if there ...
0
votes
1answer
46 views

What is the advantage of using RNN with fixed timestep length over Neural Network?

More often than not, I see RNNs being used with fixed length timesteps. So what is the difference between the following two networks? RNN with timestep length of 3 over sequence Xt. NN with inputs x(...
1
vote
1answer
18 views

Where can I download the toy benchmark dataset for RNNs?

I have read the paper: Simple Way to Initialize Recurrent Networks of Rectified Linear Units Where can I download the toy benchmark dataset for RNNs this paper mentions? I need addition problem ...
0
votes
1answer
14 views

Recurrent Neural Networks Over Multiple Documents Over Time

So in my head, I have an idea about what this architecture should look like, or at least behave, but I am having trouble implementing it. So let me describe the problem, and if anyone has an idea on ...
0
votes
1answer
8 views

Understanding Exclusive-OR predictions in Elman network

I have been reading Elman network paper, which can be found Here. in page 185, under Exclusive-OR section it was written as follows. Notice that, given the temporal structure of this sequence, it ...
0
votes
0answers
60 views

Which time step output should be used in a LSTM network?

Let's take a LSTM network with one layer and two hidden units. Let's take that the number of time steps are 4, then the input x is: \begin{align} x = \big(x\small(t),\space x\small(t-1),\space x\...
0
votes
1answer
112 views

Bidirectional GRU: validation loss stuck on plateau diverges from well performing training loss

tl;dr: What's the interpretation of the validation loss decreasing faster than training loss at first but then get stuck on a plateau earlier and stop decreasing? The accuracy behaviour is similar. ...
2
votes
0answers
37 views

GRU learns small-scale features, but misses large scales

Playing around with weather data, I have setup a simple RNN with one layer of GRUs. It is trained to recover the temperature of the next day, given weather data of the last 5 days, each with 1 hour ...
2
votes
1answer
34 views

How many Hidden Layers and Neurons should I use in an RNN?

I am very new to neural networks and machine learning and I have been making a Bitcoin price predictor to learn it. I was wondering about the number of hidden layers I'd need in a recurrent neural net ...
1
vote
1answer
335 views

Stacking LSTM layers

Can someone please tell me the difference between those stacked LSTM layers? First image is given in this question and second image is given in this article. So far what I learned about stacking LSTM ...
1
vote
2answers
63 views

Why the RNN has input shape error?

My x_train shape is (798,3) and y_train input shape is (798, 1). I am creating a RNN like this ...
0
votes
2answers
34 views

practical improvements worth trying over plain LSTM in text classification?

I have a dataset of about 1 million tweets corresponding to about 30,000 user accounts, labelled with binary data (classifying the tweet as written by a bot). With that amount of data, I could use a ...
0
votes
1answer
67 views

Why does my LSTM perform better when randomizing training subset vs. standard batch training?

I am training a simple LSTM network using Keras to predict time series values. It is a simple 2-layer LSTM. I get the best performance when I train on subsets of the training set that start at random ...
1
vote
1answer
149 views

Recurrent neural network (LSTM) dimensions error

I have data in a dataframe named ddf as follows: ...
1
vote
1answer
224 views

Shaping data for ConvLSTM for many-to-one image model

Ultimately, I am trying to obtain a binary segmentation mask for an image sequence. I have n number of image sequences, each with 500 greyscale images of size 256px by 400px. Each of these sequences ...
1
vote
0answers
22 views

Model Joint Probability of N Words Appearing Together in a Sentence

Assume that we have a large corpus of texts to train with. Given N words as input, I want to model the joint probability $p(x_1, x_2, ..., x_N)$ of these words appearing together in a sentence. More ...
3
votes
2answers
43 views

What does the one function $\mathbf{1}_{i,y^{(t)}}$ exactly mean in backward propagation of RNN in the book “Deep learning” of Bengio

It confused me for a long time what is $\mathbf{1}_{i,y^{(t)}}$ exactly mean in (10.18) below. It is in the Chapter 10 on RNN of the book LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep ...
1
vote
0answers
22 views

Addressing mechanisms in neural turing machine

In a Neural Turing Machine,why wasn't an absolute random access mechanism used?We are reading and writing based on the weighting emitted by the read/write head and this weighting is being generated by ...
4
votes
1answer
99 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
2
votes
1answer
144 views

Neural Network - distinguishing between several normalized values is impossible?

It's a common practice to normalize inputs to the neural Network. Let's assume we have a vector of activations. One of techniques, the Layer Normalization simply looks at the vector's components, ...
2
votes
0answers
69 views

Matrix multiplication issue (shapes not alligned)

I am building an RNN using numpy only and have started on the forward propagation section. However i am having some issues aligning my matrices. The issue is on this line: ...
1
vote
0answers
32 views

Simple explanation of LSTM data set and training phase

I cannot understand the training procedure of the LSTM (and other recurrent nets). My data is time series of length 2000 points. As suggested on the internet (and keras framework), this should be ...