Questions tagged [recurrent-neural-net]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
1answer
23 views

Why RNNs necessary for time series?

I got it that when using time series data, I have to use a RNN The highlight here is that the neurons receive their output again as an input, so they can take into account the previous step. But ...
0
votes
0answers
8 views

When clipping gradient is useful? [closed]

When clipping gradient is useful? It is useful for exploding gradient! But, when it is useful? for instance the weight=5 is it useful?When we are able to use clipping gradient?
1
vote
0answers
21 views

SymbolicException: Inputs to eager execution function cannot be Keras symbolic tensors

I am writing Encoder-Decoder architecture with Bahdanau Attention using tf.keras with TensorFlow 2.0. Below is my code This is working with TensorFlow 1.15 but getting the error in 2.0. you can check ...
1
vote
0answers
10 views

How to compare the complexity of different RNN cells?

I want to compare three different types of RNNs to decide which architecture can handle my data best. To do that, I want them to have the same complexity. Can I simply define the complexity by the ...
0
votes
1answer
26 views

How to apply a different Loss function to one specific Label?

I got a recurrent neural network in Keras, which classifies on 14 labels. The first label is the most important one and should be predicted with the highest accuracy. The other labels don't have to be ...
0
votes
0answers
14 views

Weather Forecasting: CNN-LSTM or ConvLSTM?

I am trying to develop a weather forecast model where satellite images (temperature, velocity field etc) are stacked over time. Since the prediction model needs to analyze both spatial features and ...
0
votes
0answers
18 views

Audio dataset preprocessing to perform cry detection

I am building a neural network to perform cry detection (i.e., binary classification of cry/non-cry situations) when capturing sound in a house environment. To do so, I performed the following steps: ...
0
votes
0answers
21 views

What does “factor computation” mean in this context?

I'm reading the paper Attention is all you need here and came along the following sentence: "Recurrent models typically factor computation along the symbol positions of the input and output ...
0
votes
0answers
30 views

Continous Learning and Shifting Patterns

I've been intensively studying neural networks which try to predict a vector based on a given input matrix. The input matrix is a N x H matrix and the output vector a N x 1 vector. The network is ...
1
vote
1answer
27 views

Structure of LSTM gates

It is my impression that a single layer LSTM architecture consists of $t$ LSTM cells that are identical duplicates, where $t$ is the number of time steps. Then there are gates within the LSTM cell. I ...
0
votes
1answer
26 views

LSTM - Incorporate word embedding in layer with multiple records in same date

I have a time series data having more than one record in a single date. Number of records in a single date is not consistent. I have 3 input features namely phrase, cost and weight. My goal is to ...
0
votes
0answers
8 views

TimeDistribution GRU with initial_state keras

I am working on a problem, where we have two inputs, a GRU layer, and an output layer: ...
1
vote
1answer
28 views

Language modelling for Spell Checker

I am working on spell checkers, I want to create a spell checker, I am confused about which model to use Word-Level modelling Character-Level modelling plus I am preferring neural networks over ...
0
votes
0answers
7 views

What techniques can I use to find position relationship in group of elements?

I have 14,000 tagged documents. These are custom forms that our employees create and fill out. I need to build a model that will be able to classify the types of each input field of the form in order ...
0
votes
0answers
8 views

Why does this RNN in tensorflow not learn

I am trying to train an RNN without using the RNN API in tensorflow (2) in Python 3.7, so the code is very basic. Something is going really wrong, but I'm not sure what it is. As a reference, I am ...
0
votes
0answers
48 views

How to prepare coordinate sequence data for machine learning classification?

I want to perform a task where the goal is to classify coordinate sequences by labels. The raw data consists of temporal log sequences for each label like this: ...
0
votes
0answers
18 views

Speed Regulation of fan using Machine Learning

Can machine learning be used for the speed regulation of fan based on the environment, how many people are present in the room and routine of a particular individual and how? How can i achieve this?
0
votes
0answers
42 views

Forecast on timeseries using known feature-values in the future

I've setup and trained an rnn-network based model on historical timeseries with 10 features. Note that I am using keras as the framework. I am happy with the results and am already using it to ...
4
votes
1answer
40 views

Do timesteps must have the same temporal distance in training a RNN?

I have a recurrent neural network with LSTM units that I want to train with batches of 6 timesteps. Each timestep is a record of a dataset and represents the temporal aggregation over 5 minutes of ...
1
vote
0answers
50 views

How to combine data having similar distribution?

I have a collection of time series data with data points of around 2 years of daily data. I am thinking of a way to increase the number of data points in it so that the neural network gets a better ...
0
votes
0answers
18 views

Training Recurrent Neural Networks with multiple time series

I have two time series, each one is a bank loan history. The rate, amount and unemployment are features. Rate and amount correspond to the loan, and unemployment is a macroeconomic variable. The ...
1
vote
0answers
17 views

Confusion in RNN terminologies?

I recently started working on RNN and LSTM and got confused with the terminologies. What is the meaning of RNN layer ? When I say RNN has 2 layer - does that mean Two RNN cell connected ...
0
votes
0answers
36 views

LSTM architecture for multivariate time series

For a multivariate time series analysis, which of the following LSTM architectures would work better and why? 1) Having two independent LSTM layers (one for the time series variable and one for the ...
1
vote
1answer
31 views

how bidirectional neural networks can be applied on time series while we do not know the future data?

I have read about bidirectional neural networks. It seems that they need input from both past and future. so lets say we are going to predict the energy use of one hour ahead having the energy use of ...
0
votes
0answers
20 views

How to use LSTM for time series data?

I've an ECG data spread over time. The duration for each data is around 3 minutes (approx 180 seconds). Each second around 200 recordings were taken. So total length for each sample is approx 36000. ...
0
votes
1answer
27 views

Contextual Spell Correction

I want to create a spell checker that corrects the spelling mistakes contextually. For example, Erroneous sentence: I want to apply for credit cart Corrected sentence: I want to apply for credit ...
0
votes
2answers
67 views

How to implement an LSTM RNN with multiple input features

EDIT: Now I didn't convert to list. I am training LSTM for multiple time-series in an array which has a structure: 450x801. There are 450 time series with each of 801 timesteps / time series. The ...
0
votes
0answers
14 views

Changing data structure in incremental learning of LSTM

This is a question which may or may not have open-ended answers. I am curious what you think and hoping to get a starting point. I am wondering what we do if we have a categorical variable in the set, ...
0
votes
0answers
24 views

LSTM training one time step at a time, and accessing prediction after each time step

I have been playing around a bit with keras LSTM but I have some confusion about it's potential applications and how to make use of it. I am new with RNNs so please correct me when I am wrong. Say ...
3
votes
2answers
174 views

Trying to understand encoder-decoder sequential models in Keras?

My understanding is that for some types of seq2seq models, you train an encoder and a decoder, and then you set aside the encoder and use only the decoder for the prediction step. For example this ...
1
vote
0answers
17 views

Confusion with initialising weights in a neural network

Here is some code that initialises weights for an RNN: ...
0
votes
0answers
26 views

How LSTM can be used to predict action and maximize sales

Hi would like to use LSTM with my dataset. most of people are using LSTM on NLP problem. In my case dataste look like this : IdCustomer | salesMonth_1 | action_1 |salesMonth_2 | action_2 |...
0
votes
0answers
15 views

What is reranking?

I am pretty new to semantic parsing and that stuff but I have to give a presentation about reranking. Can you give a good definition, what reranking does? I think it is related with the parsing ...
2
votes
1answer
55 views

What are the exact differences between Deep Learning, Deep Neural Networks, Artificial Neural Networks and further terms?

After having read some theory I am getting a bit confused about the following terms: Deep Learning Deep Neural Network Artificial Neural Network Feedforward Neural Network So, what seems clear to me ...
0
votes
0answers
10 views

Why/when is someone resetting a recurrent neural network necessary?

This is my first time implementing a recurrent neural network, and I'm confused as to why resetting the node activations is necessary. When is it necessary to reset node activations? Specifically, ...
1
vote
1answer
47 views

What is the best way Reinforcement learning, RNN or others to predict the best action we have to take to maximize sales?

I have a dataset composed of few features : customerId, actionDay1, SalesDay1, actionDay20, SalesDay20, actionDay30, SalesDay30 action can be : call email face ...
0
votes
0answers
4 views

How calculate computation time for each part of the network

I want to report how much times it takes to compute each specific part of the network in a batch (forward and backward time). For example, in this paper they've reported RNN, softmax, and optimization ...
0
votes
0answers
47 views

2019 - Bleeding edge Reinforcement Learning techniques?

I've built an RL agent using the following: Full Rainbow: Double Q-Learning (allow target network to rate the Q-score of the action selected by online network, use this score as a TD target) ...
0
votes
0answers
22 views

Time Series Forecasting with RNNs

I'm attempting to develop a recurrent model to forecast the value one step into the future (i.e., $x_{t+1}$), given its history $(x_{t-h},\cdots,x_{t})$, where $h$ is a fixed hyperparameter for the ...
0
votes
0answers
25 views

Is Elmo equivalent to Fasttext+Bi-directional GRU?

From what I have read, Elmo uses bi-directional LSTM layers to give contextual embeddings for words in a sentence. So if I use a bi-directional LSTM/GRU layer over Fasttext representations of words, ...
0
votes
0answers
14 views

Difference between globalmaxpoolin1d() and attention layer

What's the difference between globalmaxpoolin1d() and attention layer?
0
votes
3answers
61 views

How to know when to stop trainning a deep network?

I've been training several auto encoders containing two GRUs as encoder and decoder during last year. It occurred to me that ...
0
votes
0answers
39 views

In an RNN, if the gradients don't vanish for long/distant terms, won't the derivative of the error be either divergent to infinity or oscillatory?

P.S. Crosss posted here- https://stats.stackexchange.com/questions/413843/in-an-rnn-if-the-gradients-dont-vanish-for-long-distant-terms-wont-the-deriv, as I've got no answer, I'm asking here: In my ...
1
vote
0answers
85 views

Generalization of RNN/LSTM/GRU… model

Given a time-series prediction with a Recurrent Neural Network (doesn't matter if LSTM/GRU/...), a forecast might look like this: to_predict (orange) was fed to the model, predicted (purple) is the ...
1
vote
0answers
74 views

TensorFlow: how to restore pre-trained meta model and pass it's weights and biases to the optimizer?

I trained a model on a specific dataset and saved it as a meta, I want to restore the model and use its weights and biases on another dataset the code isn't mine but I'm trying to restore the ...
0
votes
0answers
12 views

How can memory networks perform well in lists/set type?

I was reading this paper about memory networks. As I understood, memory networks can give output in a word. But on Babi dataset's 'list/set' task, its accuracy was almost 80%. What have I ...
0
votes
0answers
29 views

Does an LSTM model with one hidden layer has much advantages over a RNN or NN?

Does an LSTM model with one hidden layer has much advantages over a RNN or NN? Cause the network is not deep/large
0
votes
1answer
23 views

How can I train a many-to-one RNN with an array of 2D matrices?

I have eye tracking data for every word of a novel. Features for every word is given separately. I want to take groups of 100 words to make a sample and then use each of these samples as a single ...
0
votes
0answers
14 views

Why are reservoir computer so useful for hardware implementations

I often read (e.g. here or in this question) that Reservoir Computer (RC) are useful in the field of Neuromorphic Computing where they can serve as efficient implementations of neural networks in ...
0
votes
0answers
11 views

What will go wrong if we apply linear or other types of regression to translate sentences between two languages?

Disclaimer: I asked the question at https://stats.stackexchange.com/questions/408463/what-will-go-wrong-if-we-apply-linear-or-other-types-of-regression-to-translate, but didn't get any response, so I'...