Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
0
votes
1answer
22 views

What is the input of LSTM network?

Hello I am trying to understand LSTMs but have a few problems: What is the input? Since LSTM is seq2seq I would think it is a sequence of words, but in a Codecademy lesson is mentioned that each ...
1
vote
0answers
7 views

why would you mask out padded activations from the training loss?

I've followed taming-lstm for training a LSTM model on a NLP task in batches with various sentence lengths. One of his main points is: Trick 3: Mask out network outputs we don’t want to consider in ...
0
votes
0answers
21 views

Making Keras RNN to proceed input sequence step by step

I'm currently trying to create a neural network for playing Tetris. I'm using evolutionary algorythms for it's learning, so the behavior that I need to get from the neural network is the following: ...
0
votes
0answers
37 views

Regression with LSTM network: use multiple time series as input

I've spent a few days on this and am starting to think I'm missing the obvious solution as this doesn't seem like a very uncommon problem. As an example dataset: I have 100 measurements with each a ...
0
votes
0answers
14 views

Regarding the application of RNN on new data

Suppose the two univariate time series $X_{1,T}=(x_1, x_2, ..., x_T)$ and $Y_{1,T}=(y_1, y_2, ..., y_T)$. The next step would be to train an RNN or LSTM with input $X_{1,T}$ and output $Y_{1,T}$, in ...
0
votes
0answers
16 views

Derivation of HiddenState wrt Output of LSTM

I'm busy trying to understand the math behind LSTM RNN's. In most of the math tutorials that I've found the derivations (Backpropagation) don't consider a dense layer before the output, instead they ...
0
votes
0answers
17 views

Passing data to RNN with Sliding window approach

I having hard time with LSTM's and RNN so my apologies if this question sounds like a very basic question. I would appreciate if you can help in any way. I am trying to train my RNN with LSTMs, but I ...
0
votes
0answers
26 views

How to implement a Multivariate multi-site application in LSTM?

I am trying to make a multivariate multi-site classification LSTM model using Keras. I have followed this tutorial from Jason Brownlee: https://machinelearningmastery.com/multivariate-time-series-...
0
votes
0answers
16 views

How to improve a LSTM self-attention model given the absence of overfitting

I am doing a binary classification on time series data. Class 0 is a single class but class 1 is actually a combination of 7 different classes. My objective to classify class 0 from other classes. The ...
0
votes
0answers
10 views

Share the experiences of regularization of a LSTM model

There are five parameters from an LSTM layer for regularization if I am correct. To deal with overfitting, I would start with reducing the layers reducing the hidden units Applying dropout or ...
1
vote
1answer
22 views

Number of parameters in Simple RNNs

Please, I am stuck, I can not understand the number of parameters of a simple RNN, here the example and the model summary. the example is simple: ...
0
votes
0answers
12 views

Please verify my RNN/LSTM multilingual multi class text classifier approach

Can you please verify my approach for a multilingual multi class text classifier using RNN/LSTM? The dataset consists of authors (labels, response) and their letters (input data). The letters can be ...
0
votes
0answers
8 views

Series Through a GPU's Window To, For Each Item, Output a Prediction and Retrain?

Perhaps I'm missing something obvious but I've not run across a Keras or PyTorch example of online training and series prediction loop implemented on a GPU with these (seemingly obvious) ...
1
vote
0answers
13 views

Advice on using Recurrent Neural Networks for non-time series dataset

I'm testing different machine learning algorithms for predicting week-to-week fantasy football scores for individual players. For those who don't know, fantasy football is a game in which players pick ...
0
votes
0answers
15 views

How far into the future can I forecast a time-series with an LSTM and strongly seasonal data

I am working on a Sequence-to-Sequence + Attention model for some time-series data. Now I have a really long time series, basically 40 years of daily observations for multiple sensors. The data itself ...
0
votes
0answers
41 views

ML Algorithm or approach to solve timeseries selection

I am fairly new to ML and I'm working on a problem, but not sure which algorithm to choose. The dataset contains a set of incremental time-series events, in consistent and set intervals, with a list ...
0
votes
1answer
20 views

Understanding projection layer for BLSTM

In many research papers there are 'projection layers' related to BLSTM layers. For example, from here: "we trained an 8-layer BLSTM encoder including 320 cells in each layer and direction, and ...
1
vote
1answer
23 views

Long range forecasting with sequence-to-sequence models

I have a task where I want to forecast daily observations for 1 year or 2 years in advance at multiple locations--so 365 or 730 days in advance. I actually have a pretty good dataset, meaning daily ...
0
votes
1answer
28 views

Optimal input setup for character-level text classification RNN

I want to classify 500-character long text samples as to whether they look like natural language using a character-level RNN. I'm unsure as to the best way to feed the input to the RNN. Here are two ...
0
votes
1answer
54 views

Are there any control-flow/conditional statements in AI/ML models?

I was recently asked this during an interview. When we write a C program, it has a control-flow in the form of conditional statements like if, ...
1
vote
2answers
25 views

Can I create a layer with multiple rnn cell ? [question about a paper]

I am trying to implement https://dl.acm.org/doi/pdf/10.1145/3269206.3271794 . Structure: As it said: In particular, we integrate the embedding vectors learned from each individual recurrent encoder ...
0
votes
0answers
8 views

Help understanding input to biaxial network for generating music

I am reading Composing Music With Recurrent Neural Networks by Daniel D. Johnson. But I am really confused about the input passed to this network. If we pass notes of music along the time axis, then ...
0
votes
0answers
25 views

Is padding the right way to allow your model to make prediction with test sequences of shorter lengths?

Say I have a RNN-lstm encoder-decoder model trained on fixed timesteps (no padding when training, all sequences are treated as if having the same lengths). My testing criteria requires me to provide ...
0
votes
2answers
40 views

Why use gradient descent on Deep Nets / RNNs when cost function is not convex?

Why do we use gradient descent on very non-convex loss functions such as in Deep nets / RNNs rather than a heuristic search (genetic algorithms, simulated annealing, etc)?
0
votes
0answers
6 views

Make a binary classification RNN to only focus on “malicious” words to make the model more robust? (Non negative model)

There are some works that make a model robust against attacks, by making the model "focus" on only malicious features, therefore addition of benign features will not affect the outcome of ...
0
votes
0answers
15 views

What does the inital_state parameter in the GRU call arguments do?

Does the inital_state parameter in the GRU call arguments, specify the inital state of the hidden state, that is, $h_t$ or the weights?
0
votes
1answer
21 views

How is the hidden state of a GRU initialized

This is a GRU. Now, what will be the value of $h_t$, at $t$=$0$. That is, what will be the value of the hidden state at just the starting?
0
votes
0answers
17 views

Number of Parameters in Multidimensional rnn

I have the following equation for the hidden unit in a Multidimensional RNN and I need to calculate the total number of trainable parameters: $$h^{i,j} = \sigma (W_{in}x^{i,j} + W_{left}h^{i-1,j} + ...
6
votes
2answers
487 views

How is it possible for RNN to do sentiment analysis?

I'm wondering how RNN can be used when doing sentiment analysis. It seems that the characteristic of RNN is to remember what appeared in the past and determine the value of the present (future), but I ...
0
votes
0answers
30 views

Ideal framing of data from LSTM + adding static features

I'm dealing with a problem that I'll try to simplify here : You have a land where you can plant seeds and water them. On each day, you have the total area being watered and you have area that has just ...
1
vote
0answers
28 views

Implementation of the LSTM using Keras in R with multiple outputs

I'm implementing the LSTM based on this tutorial (https://blogs.rstudio.com/ai/posts/2017-12-20-time-series-forecasting-with-recurrent-neural-networks/), but the example consider multiple inputs with ...
2
votes
0answers
41 views

Machine learning for circular sequences

My data are sequences of real numbers $a_0,a_1,...,a_{n-1}$. The length of a sequence is fixed and equals $n$. Each sequence is mapped to a real number $y$ and I want to predict $y$ given the sequence....
1
vote
0answers
22 views

What information does Hidden State and Cell State carry? [duplicate]

What information does Hidden State and Cell State carry? Do they carry the summary of the input sequence or what?
3
votes
0answers
49 views

How is the input gate in the LSTM learn?

How is the input gate neural network trained what to remember by propagating the error rate from predicting the next word in the language model? How does it help it to learn if it remembered the right ...
4
votes
1answer
1k views

Difference between cell state and hidden state

LSTM cells consist of two types of states, the cell state and hidden state. How do cell and hidden states differ, in terms of their functionality? What information do they carry?
0
votes
0answers
15 views

How to predict the temperature for new points?

Assume that we are going to predict the temperature change of a specific point in a liquid over time. We have the following as our observation: We have the information about 200 different points in ...
1
vote
1answer
48 views

LSTM Target Is Also One of It's Inputs?

I have two input arrays that include both historical and forecasted data, and one input array that is only historical. I'm trying to predict (or "forecast") the latter array given the ...
2
votes
1answer
223 views

Difference between Jordan, Elman and normal RNN

As far as I know for history, the Jordan network was proposed first in 1986 as a form of RNN with this diagram: Actually, this is the solution that makes sense when thinking about sequence data that ...
2
votes
2answers
54 views

What if we input sequence data to feedforward network?

One main advantage of RNN is the ability to take input of variable length like the case of sequences. However, what if we neglected this case and assumed some applications that may accept some fixed ...
1
vote
1answer
38 views

How does “one-to-many” RNNs work?

I recently came across an article about RNNs here. Which describes different types of RNNs like: The first figure makes sense. A regular feedforward network. The second is a big question for me. Is ...
0
votes
0answers
30 views

Google Trax's GRU layer

I am learning about Trax for the implementation of GRU and LSTMs. Their documentation says that a GRU layer in Trax can only accept a number of hidden units equal to the number of elements in the ...
1
vote
0answers
20 views

state transition classification on terminal state

I have data on a unit $i$ which enters an entry state $S_0$. This unit has some covariates $x_i$ I would like to predict the probability the unit will reach the terminal state $S_{pos}$ or $S_{neg}$. ...
0
votes
0answers
13 views

Time duration weighted recurrent neural network

Suppose the input time-series feature is $\vec{X}=[\mathbf{x_0},\mathbf{x_1},...\mathbf{x_T}]$, where at each time step $t\in[0,...,T]$, feature $\mathbf{x_t}$ is a vector with dimension $n$. Typical ...
0
votes
0answers
19 views

Architecture for sequential data

I am seeking advice on a solution I am building as a part of my data science course. I have to build a web application that solves a problem created by COVID-19. My idea - For the first stage of my ...
0
votes
1answer
40 views

How can I picture an unfolded RNN as a normal Feed Forward Network?

I am currently working on a Transformer architecture. Trying to picture an RNN (or Encoder) as a normal Feed Forward network really confused me after looking at the following image in an article: (...
0
votes
1answer
28 views

LSTM / GRU weights during test time

I am working on a historic time series dataset and using RNN, LSTM, GRU models, and I didn't find an answer if in test time, the h (or h, c) weights should be zeors for each batch? If the weights ...
0
votes
0answers
22 views

Properties of embeddings that work best in recurrent layers

Let's assume that you have a simple recurrent layer that works on top of sequences of dense embeddings, predicting next embedding. The embeddings are generated by an autoencoder. What properties ...
2
votes
1answer
27 views

GRU and LSTM does not “take risk” predicting

I tested LSTM and GRU models to predict the exchange rate between currencies. I do not take the raw price but a the delta with the previous day, so the data is stationnary around zero. My problem is ...
0
votes
1answer
35 views

best approach to embed random length sequences of words as a fixed size vector without having a maximum length? [closed]

I have a dataset of sentences in a non-English language like: word1 word2 word3 word62 word5 word1 word2 and the length of each sentence is not fixed. Now, I want to represent each sentence as a ...
2
votes
2answers
1k views

Can we use BERT for only word embedding and then use SVM/RNN to do intent classification?

According to this article, "Systems used for intent classification contain the following two components: Word embedding, and a classifier." This article also evaluated BERT+SVM and Word2Vec+...

1
2 3 4 5
10