Questions tagged [recurrent-neural-net]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
6
votes
1answer
195 views

A the end of a big DS project, should I make trained models available on GitHub?

I almost completed two big Data Science personal projects based on Deep Learning. They are the fanciest models I've implemented up to now, and I'm pushing all my code on GitHub. Do you advice to ...
0
votes
0answers
10 views

Long sequence prediction with model trained on short sequence

I'll start with a specific example. I would like to train model which predict vector of [0-1]. Values close to 1 on specific range indicates that in those timesteps is particular activation word (...
1
vote
1answer
23 views

How to code a simple forward propagation of recurrent neural networks?

I know the theory behind recurrent neural networks or RNN but I am confused about its implementation. This is an rnn equation I got from the web, I tried to code the forward propagation alone in ...
0
votes
0answers
19 views

Summarize events per ID

Data: Each corresponds to an event (a person's visit to the hospital, as an example). I have a series of data associated with this event (duration of visit, motive, etc...). Objective: Summarize the ...
0
votes
0answers
11 views

Increase dimension of RNN LSTM cell in Keras

I want to increase amount of recurrent weights in rnn or lstm cell. The idea is that RNN neuron takes prevois output as input. I want to increase amount of previous values taken as input. If you ...
1
vote
2answers
22 views

Which input to use when generating a new sequence

I want to use sequence-to-sequence architecture to generate sequences. My data has such structure ...
4
votes
2answers
89 views

RNN in pseudo-code

A few years ago, I understood the classical MLP neural network much better when I wrote an implementation from scratch (using only Python + Numpy, without using tensorflow). Now I'd like to do the ...
1
vote
0answers
15 views

Credit attribution for prediction in recurrent neural nets

Consider a recurrent neural net, which has access two inputs sequences x1,x2,x3,x4.... and s1,s2,s3,s4... It emits a predictions p1,p2,p3,p4.... where ...
2
votes
0answers
15 views

Basic questions about hamming network

I'm reading Hogan et al's Neural Network Design book very closely. I have run into a couple of questions about its presentation on Hamming networks. In particular, it says: The next network we ...
0
votes
0answers
31 views

Generalised Estimating Equation (GEE) vs. Recurrent Neural Network (RNN)

Has anyone looked into or know what is the difference between a GEE model and an RNN model in terms of what these two models are doing? Apart from the differences in structure of these two models ...
0
votes
0answers
11 views

Multimodal vs recurrent network for video-based FER

I am wondering what types of architectures are best to explore for doing video-based facial emotion recognition. Some application feature using an architecture spatial-temporal architecture of 2 CNNs, ...
1
vote
1answer
20 views

How to return states of LSTM in MXNet?

I used MXNet previously to beat keras+tensorflow accuracy in CNN regression models. Now I am trying to implement LSTM, which in keras runs fine: ...
0
votes
1answer
33 views

What is the output of multivariate LSTM model?

I am currently trying to build an LSTM model by using multivariate inputs, but I don't understand what exact output I am predicting. I am currently using 5 features in the data i.e. 'Time', 'Avg CPU ...
1
vote
0answers
18 views

Implementing an RNN on multiple text sources

I want to implement an RNN to generate a new text based on many examples of existing texts of a certain format in the training data. The type of texts in the training data consists of 3 segments, ...
1
vote
1answer
182 views

What is a 'hidden state' in BERT output?

I'm trying to understand the workings and output of BERT, and I'm wondering how/why each layer of BERT has a 'hidden state'. I understand what RNN's have a 'hidden state' that gets passed to each ...
2
votes
2answers
92 views

What is the role of $W_{ax}, W_{aa}, W_{ay}$ in forward propagation in RNN? Are they hyperparameters? Why are they needed?

In RNN introduction in Coursera sequence model course, the following formula for forward propagation in RNN was introduced. What exactly is the role of $W_{ax}, W_{aa}, W_{ay}$? What do they do? In ...
1
vote
0answers
8 views

Scipy.optimize getting back x0 for optimization of input to recurrent neural network

I need help with this optimization problem which is either not getting solved at all or is taking a copious amount of time. I am trying to find optimized input to an RNN (GRU) model of a process ...
2
votes
0answers
21 views

Masking seems not working for missing values problem in LSTM

I am trying to use LSTM to predict time series in keras. My input data shape is (1000,6,1)(samples,timesteps,features). There is some missing data in different timesteps. For example,[2,1,1]=NaN,[3,4,...
1
vote
1answer
26 views

How can I compare my regressors?

I am trying to build a regressor for a dataset which gives info about students' school performance and the probability of getting admitted in the University of their choice. The first 5 observations ...
1
vote
1answer
22 views

How does backpropagation work with averaging layers?

I'm studying Word2Vec algorithm, and so far i understood that, in the case of input context bigger than 1 (so multiple words) we have our hidden layer that performs averaging between the inputs (as ...
1
vote
1answer
176 views

A cross-entropy loss explanation in simple words

Suppose I build a FNN model. The last layer is a classification layer with softmax activation. A cross-entropy loss is used to classify a problems, such as logistic regression. How would I calculate ...
0
votes
0answers
51 views

how to create custom data-set for “text-detection-ctpn mode” for training the model to localize text in particular field of interest?

how to create custom data-set for "text-detection-ctpn mode"? Git link: "https://github.com/eragonruan/text-detection-ctpn" Query1. How the coordinates ...
1
vote
1answer
21 views

Efficient recurrent network for sequences of varying length

Suppose I have a bunch of sequences of varying lenghts. The absolute majority of them are short, just a few dozens items long. However, very few of them are significantly longer - more than a hundred ...
2
votes
1answer
36 views

Why RNNs necessary for time series?

I got it that when using time series data, I have to use a RNN The highlight here is that the neurons receive their output again as an input, so they can take into account the previous step. But ...
1
vote
0answers
498 views

SymbolicException: Inputs to eager execution function cannot be Keras symbolic tensors

I am writing Encoder-Decoder architecture with Bahdanau Attention using tf.keras with TensorFlow 2.0. Below is my code This is working with TensorFlow 1.15 but getting the error in 2.0. you can check ...
1
vote
0answers
15 views

How to compare the complexity of different RNN cells?

I want to compare three different types of RNNs to decide which architecture can handle my data best. To do that, I want them to have the same complexity. Can I simply define the complexity by the ...
0
votes
2answers
43 views

How to apply a different Loss function to one specific Label?

I got a recurrent neural network in Keras, which classifies on 14 labels. The first label is the most important one and should be predicted with the highest accuracy. The other labels don't have to be ...
0
votes
0answers
186 views

Weather Forecasting: CNN-LSTM or ConvLSTM?

I am trying to develop a weather forecast model where satellite images (temperature, velocity field etc) are stacked over time. Since the prediction model needs to analyze both spatial features and ...
0
votes
0answers
41 views

Audio dataset preprocessing to perform cry detection

I am building a neural network to perform cry detection (i.e., binary classification of cry/non-cry situations) when capturing sound in a house environment. To do so, I performed the following steps: ...
0
votes
0answers
24 views

What does “factor computation” mean in this context?

I'm reading the paper Attention is all you need here and came along the following sentence: "Recurrent models typically factor computation along the symbol positions of the input and output ...
0
votes
0answers
31 views

Continous Learning and Shifting Patterns

I've been intensively studying neural networks which try to predict a vector based on a given input matrix. The input matrix is a N x H matrix and the output vector a N x 1 vector. The network is ...
1
vote
1answer
95 views

Structure of LSTM gates

It is my impression that a single layer LSTM architecture consists of $t$ LSTM cells that are identical duplicates, where $t$ is the number of time steps. Then there are gates within the LSTM cell. I ...
0
votes
1answer
38 views

LSTM - Incorporate word embedding in layer with multiple records in same date

I have a time series data having more than one record in a single date. Number of records in a single date is not consistent. I have 3 input features namely phrase, cost and weight. My goal is to ...
0
votes
0answers
20 views

TimeDistribution GRU with initial_state keras

I am working on a problem, where we have two inputs, a GRU layer, and an output layer: ...
1
vote
1answer
31 views

Language modelling for Spell Checker

I am working on spell checkers, I want to create a spell checker, I am confused about which model to use Word-Level modelling Character-Level modelling plus I am preferring neural networks over ...
0
votes
0answers
8 views

What techniques can I use to find position relationship in group of elements?

I have 14,000 tagged documents. These are custom forms that our employees create and fill out. I need to build a model that will be able to classify the types of each input field of the form in order ...
0
votes
0answers
10 views

Why does this RNN in tensorflow not learn

I am trying to train an RNN without using the RNN API in tensorflow (2) in Python 3.7, so the code is very basic. Something is going really wrong, but I'm not sure what it is. As a reference, I am ...
0
votes
0answers
91 views

How to prepare coordinate sequence data for machine learning classification?

I want to perform a task where the goal is to classify coordinate sequences by labels. The raw data consists of temporal log sequences for each label like this: ...
0
votes
0answers
26 views

Speed Regulation of fan using Machine Learning

Can machine learning be used for the speed regulation of fan based on the environment, how many people are present in the room and routine of a particular individual and how? How can i achieve this?
0
votes
0answers
42 views

Forecast on timeseries using known feature-values in the future

I've setup and trained an rnn-network based model on historical timeseries with 10 features. Note that I am using keras as the framework. I am happy with the results and am already using it to ...
4
votes
1answer
50 views

Do timesteps must have the same temporal distance in training a RNN?

I have a recurrent neural network with LSTM units that I want to train with batches of 6 timesteps. Each timestep is a record of a dataset and represents the temporal aggregation over 5 minutes of ...
1
vote
0answers
54 views

How to combine data having similar distribution?

I have a collection of time series data with data points of around 2 years of daily data. I am thinking of a way to increase the number of data points in it so that the neural network gets a better ...
0
votes
0answers
41 views

Training Recurrent Neural Networks with multiple time series

I have two time series, each one is a bank loan history. The rate, amount and unemployment are features. Rate and amount correspond to the loan, and unemployment is a macroeconomic variable. The ...
1
vote
0answers
24 views

Confusion in RNN terminologies?

I recently started working on RNN and LSTM and got confused with the terminologies. What is the meaning of RNN layer ? When I say RNN has 2 layer - does that mean Two RNN cell connected ...
0
votes
0answers
44 views

LSTM architecture for multivariate time series

For a multivariate time series analysis, which of the following LSTM architectures would work better and why? 1) Having two independent LSTM layers (one for the time series variable and one for the ...
1
vote
1answer
59 views

how bidirectional neural networks can be applied on time series while we do not know the future data?

I have read about bidirectional neural networks. It seems that they need input from both past and future. so lets say we are going to predict the energy use of one hour ahead having the energy use of ...
0
votes
0answers
22 views

How to use LSTM for time series data?

I've an ECG data spread over time. The duration for each data is around 3 minutes (approx 180 seconds). Each second around 200 recordings were taken. So total length for each sample is approx 36000. ...
0
votes
1answer
28 views

Contextual Spell Correction

I want to create a spell checker that corrects the spelling mistakes contextually. For example, Erroneous sentence: I want to apply for credit cart Corrected sentence: I want to apply for credit ...
0
votes
2answers
197 views

How to implement an LSTM RNN with multiple input features

EDIT: Now I didn't convert to list. I am training LSTM for multiple time-series in an array which has a structure: 450x801. There are 450 time series with each of 801 timesteps / time series. The ...
0
votes
0answers
21 views

Changing data structure in incremental learning of LSTM

This is a question which may or may not have open-ended answers. I am curious what you think and hoping to get a starting point. I am wondering what we do if we have a categorical variable in the set, ...