Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [recurrent-neural-net]

The tag has no usage guidance.

0
votes
1answer
13 views

How can I train a many-to-one RNN with an array of 2D matrices?

I have eye tracking data for every word of a novel. Features for every word is given separately. I want to take groups of 100 words to make a sample and then use each of these samples as a single ...
0
votes
0answers
9 views

Why are reservoir computer so useful for hardware implementations

I often read (e.g. here or in this question) that Reservoir Computer (RC) are useful in the field of Neuromorphic Computing where they can serve as efficient implementations of neural networks in ...
0
votes
0answers
11 views

What will go wrong if we apply linear or other types of regression to translate sentences between two languages?

Disclaimer: I asked the question at https://stats.stackexchange.com/questions/408463/what-will-go-wrong-if-we-apply-linear-or-other-types-of-regression-to-translate, but didn't get any response, so I'...
2
votes
0answers
27 views
0
votes
0answers
19 views

What's the difference between hidden layer size and sequence length in RNN and LSTM?

I have been exploring RNNs in keras implementations. In the LSTM layer we have to provide a hidden layer size and also a sequence length. My question is, what does hidden layer size correspond to and ...
1
vote
0answers
36 views

How to apply an RNN to forecast non-stationary time series?

Is it possible to predict a time series which is non-stationary, in the sense that, the dependent variable Y have an increasing trend? Therefore, the highest value of $Y$ in the training set may be ...
0
votes
1answer
32 views

LSTM input and output for sentiment analysis

I'm studying this LSTM network: https://www.kaggle.com/paoloripamonti/twitter-sentiment-analysis ...
0
votes
1answer
13 views

Why don't we gradually update the activation parameters in RNN from one activation to the next as the network is learning more?

I'm very new to (unidirectional, vanilla) RNN and sequence modeling in general, and all I understood about the motivation on having the connection between two successive hidden layers/activation is ...
0
votes
0answers
60 views

LSTM Predict values out of test

I'm trying to predic stock values from a dataset, for example: Google stock. I have this easy model. ...
2
votes
1answer
20 views

How to perform polynomial landmark detection with deep learning

I am trying to build a system to segment vehicles using a deep convolutional neural network. I am familiar with predicting a set amount of points (i.e. ending a neural architecture with a Dense layer ...
0
votes
0answers
15 views

Strategy for “forcing” number of labels in seq2seq predictions with Keras?

I'm trying to train a seq2seq model that for every timestep in a given timeseries sample will output 1 of 6 possible labels. Furthermore, the training data is constructed in such a way that Each ...
1
vote
0answers
26 views

One hot encoding as input to recurrent neural networks

I'm trying to predict next label in a pattern based on previous labels using recurrent neural network. In total I have 100 labels Example of input pattern: ...
0
votes
0answers
26 views

Using an RNN to predict fantasy football results

So I have a couple questions about the design of a neural network. I'm trying to create a neural network to predict the number of fantasy points a player will score in a given week. First of all, I ...
1
vote
0answers
14 views

Sequence classification using oneClass SVM

In the code below, I'm using a sequence to sequence approach as a prediction model for anomaly detection, The data set I'm working with is ADFA-LD. The training phase is done using only normal ...
1
vote
1answer
88 views

How to feed a table per timestamp to LSTM neural network?

I have a time-series dataframe like this ...
0
votes
0answers
23 views

What encoding to use for my musical vectors?

I'm trying to build a music recommendations system using an encoder-decoder sequence-to-sequence architecture using keras. My dataset comprises of playlists containing songs represented as a 13-...
1
vote
1answer
24 views

Working ofLSTM with multiple Units - NER

I am trying to understand working of LSTM networks and kind of not clear about how different neurons in a cell interact each other. I had a look at a similar question, but still not clear about few ...
1
vote
0answers
20 views

Adding context in a sequence to sequence problem

The encoder of a seq2seq model is meant to generate a conditioning context for the decoder, as mentioned here A RNN layer (or stack thereof) acts as "encoder": it processes the input sequence and ...
1
vote
0answers
14 views

Why is MLP working similar to RNN for text generation

I was trying to perform text generation using only a character level feed-forward neural network after having followed this tutorial which uses LSTM. I one-hot encoded the characters of my corpus ...
0
votes
0answers
17 views

My question is about dependency between hidden states for Back Propagation Through Time in RNN

In one video lecture, professor Ali Ghodsi of University of Waterloo says that the first node of S(t)(hidden state of RNN at time t) has an effect only on the first node of S(t+1)(hidden state of RNN ...
1
vote
1answer
122 views

how to apply MC dropout to an LSTM network keras

I have a simple LSTM network developped using keras: ...
2
votes
2answers
194 views

Unnormalized Log Probability - RNN

I am going through the deep learning book by Goodfellow. In the RNN section I am stuck with the following: RNN is defined like following: And the equations are : Now the $O^{(t)}$ above is ...
1
vote
1answer
30 views

How to estimate the not available observation in time series data?

Suppose, I have a 30 seconds time-step observations of sports data, in some of the intervals the game was partially/fully stopped. I'm trying to prep the data for a time series analysis. Is it ...
2
votes
1answer
32 views

How to create a language translator from scratch?

I want to create a translator which can translate English, Korean and Tamil sentences into English sentence, I tried googletrans but is there any way to create something better than that using DL and ...
1
vote
0answers
18 views

Reinforcement learning - generating a matrix of continuous values with varying size for test data generation

Currently, I am using RL A3C algorithm for test data generation, where for a set of 30 functions written in C (mostly basic algorithms like Prime number checks, triangle validity, etc.) I try to ...
0
votes
1answer
41 views

Accuracy and Loss in MLP

I am trying to explore models for predicting whether the a team will win or lose based on features about the team and their opponent. My training data is 15k samples with 760 numerical features. Each ...
1
vote
1answer
33 views

principles of time series analysis by neural network models

I can understand for speech signals, words are correlated and therefore one should have a reason to believe that recurring NNs or LSTMs could predict by running some complex algorithm with weights and ...
0
votes
0answers
20 views

Is this code correct for a sequential model for time series pattern prediction Keras

posting this to stack exchange DS as I have also seen people answering keras related questions here! I have a question about pre-processing data in order to enter it into a sequential model in keras ...
0
votes
0answers
28 views

How to training the recurrent recommender system with LSTM?

Recently, I read a paper about recurrent recommender system, I am very curious about how it training its network. Assume I have the Netflix dataset as ...
0
votes
1answer
37 views

Is the LSTM share the same model parameter in each block?

I learn the LSTM recently, and a little bit confuse about the model parameters about LSTM, The follow is the LSTM structure And it is equation as (I slightly ignore the bias in each equation): $$...
0
votes
1answer
44 views

ML technique to predict next performance anomaly

I would like a direction for a ML technique to predict when an anomaly will occur in a system, like when the CPU use differs from a normal state. My data consists of system metrics collected from AWS ...
1
vote
2answers
189 views
2
votes
1answer
221 views

How to reshape data for LSTM training in multivariate sequence prediction

I want to build an LSTM model for customer behaviour. It's the first time for me working on a timeseries, so some concepts are not clear to me at all. My prediction problem is multidimensional, ...
0
votes
0answers
26 views

Neural Network Architecture for batch of time series data

Let's say I have a data set which is a 2-Dimensional Matrix as the input and I want to predict either 0 or 1 with regard to the entire 2-D matrix. Now each row in the 2-D matrix is a time series, i.e....
3
votes
0answers
51 views

LSTM Long Term Dependencies Keras

I am familiar with the LSTM unit (memory cell, forget gate, output gate etc) however I am struggling to see how this links to the LSTM implementation in Keras. In Keras the input data structure for X ...
1
vote
0answers
24 views

What should the size of the decoder output be in a sequence to sequence model

In a sequence to sequence model, a lot of the tutorials I have read state that the decoder target length should be the same as the encoder input length (https://blog.keras.io/building-autoencoders-in-...
1
vote
2answers
44 views

Is there a disadvantage to letting a model train for a large number of epochs?

I created a model to solve a time series forecasting problem. I had a limited amount of time series with which I could train the model therefore I decided to augment the data. The data augmentation ...
2
votes
1answer
1k views

How to determine feature importance in a neural network?

I have a neural network to solve a time series forecasting problem. It is a sequence-to-sequence neural network and currently it is trained on samples each with ten features. The performance of the ...
3
votes
1answer
65 views

Is an Arma model equivalent to a 1-layer Recurrent Neural Network without activation function?

Given a time series $f(t)$ to forecast, let us consider an Arma model of the form: $$ f(t) = c + \sum_{i=1}^p a_i f(t-i) + e(t) + \sum_{j=1}^q b_j e(t-j) $$ where $e(t)$ are the forecast errors. On ...
0
votes
1answer
52 views

Architecture for linear regression with variable input where each input is n-sized one-hot encoded

I am relatively new to deep learning (got some experience with CNNs in PyTorch), and I am not sure how to tackle the following idea. I want to parse a sentence, e.g. I like trees., one-hot encoded the ...
1
vote
0answers
37 views

Mini-batches with sequential data

I am a little bit confused. When using mini-batches, it is a good idea to shuffle. This will not work if the training examples are dependent on each other, e.g. 5 minute voltage measurement data, ...
0
votes
0answers
141 views

Multivariate LSTM RMSE value is getting very high

I want to predict a time series with multiple variables. I am using Keras's LSTM class. Here is my data set description : I want to predict var1(t-1) and my X variables are var3(t-1) , var4(t-1) , ...
5
votes
2answers
1k views

Validation loss is not decreasing

I am trying to train a LSTM model. Here is train and validation loss graph. Is this model suffering from overfitting problem ?
1
vote
1answer
148 views

Need to make an multivariate RNN, confused about input shape?

So I've seen this: Keras LSTM with 1D time series And this: Multi-dimentional and multivariate Time-Series forecast (RNN/LSTM) Keras But I still don't quite get it. I have many, many, many accountIDs,...
2
votes
1answer
53 views

Sequential Modelling: Multiple Sequence to One or Sequence to Sequence

Suppose I have a single sequence of $x_1, x_2, ..., x_n$ and corresponding labels $y_1, y_2, ..., y_n$. An example would be a person makes website visits $x_i$ and the label $y_i$ tells us if there ...
0
votes
1answer
67 views

What is the advantage of using RNN with fixed timestep length over Neural Network?

More often than not, I see RNNs being used with fixed length timesteps. So what is the difference between the following two networks? RNN with timestep length of 3 over sequence Xt. NN with inputs x(...
1
vote
1answer
19 views

Where can I download the toy benchmark dataset for RNNs?

I have read the paper: Simple Way to Initialize Recurrent Networks of Rectified Linear Units Where can I download the toy benchmark dataset for RNNs this paper mentions? I need addition problem ...
0
votes
1answer
16 views

Recurrent Neural Networks Over Multiple Documents Over Time

So in my head, I have an idea about what this architecture should look like, or at least behave, but I am having trouble implementing it. So let me describe the problem, and if anyone has an idea on ...
0
votes
1answer
13 views

Understanding Exclusive-OR predictions in Elman network

I have been reading Elman network paper, which can be found Here. in page 185, under Exclusive-OR section it was written as follows. Notice that, given the temporal structure of this sequence, it ...
0
votes
0answers
135 views

Which time step output should be used in a LSTM network?

Let's take a LSTM network with one layer and two hidden units. Let's take that the number of time steps are 4, then the input x is: \begin{align} x = \big(x\small(t),\space x\small(t-1),\space x\...