Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
1
vote
0answers
8 views

LSTM low training/validation error but really bad predictions

I'm building a LSTM model to create an automatic drums composer. I'm following this post: LSTM Metallica I've built my model and done all the enconding, I was able to emulate the behavior of the ...
-1
votes
0answers
24 views

Accuracy during the test is very low and the result is similar to a head or tail

First I'm sorry, I don't know Python but I need to get these positive results. I need to find out where I'm going wrong, because such a low accuracy and bad final test. ...
1
vote
0answers
9 views

Custom GRU With 3D Spatial Convolution Layer In Keras

I am trying to implement a custom GRU model that is shown in this paper 3D-R2N2 The GRU pipeline looks like: The original implementation is theano based and I am trying to apply the model in tf2/...
0
votes
0answers
8 views

Why is my time series model predicting strange results?

I am trying to predict some time-series data. The output data predicts two numbers (one that's usually greater than 1 and another that is usually less than 1). I've plotted about 800 samples where the ...
0
votes
2answers
22 views

Why are RNNs used in some computer vision problems?

I am learning computer vision. When I was going through implementations of various computer vision projects, some OCR problems used GRU or LSTM, while some did not. I understand that RNNs are used ...
-1
votes
0answers
19 views

LSTM (Long Short-Term Memory) network usecase

I am new to LSTM and trying to put in a real life implementation, but not sure whether LSTM suits well. The use case is as following: There are many warehouses in different regions. Hourly, trucks ...
0
votes
0answers
13 views

predict the length of a sentence

I imagine setting up training data with this format: (sentences would typically be 10 - 40 words) For training: ...
0
votes
1answer
15 views

what is the difference between positional vector and attention vector used in transformer model?

what is the difference between positional vector and attention vector used in transformer model ? , i saw a video in youtue and the defintion for positional vector was give as :* "vector that ...
0
votes
1answer
97 views

ValueError: Error when checking input: expected the_input to have 3 dimensions, but got array with shape (14174, 1)

hope you're all doing good ! I am working on Automatic Speech Recognition with Python with the LibriSpeech Dataset. After preprocessing the audios data and applying an "MFCC featurizing" I ...
1
vote
0answers
29 views

Streaming sequence detection (Binary Classification) LSTM/GRU

I am currently trying to implement a model which can detect a specific sequence according to the training data which looks like the following: ...
1
vote
1answer
24 views

How similar is Adam optimization and Gradient clipping?

According to the Adam optimization update rule: $$m \leftarrow \beta_1 m + (1 - \beta_1)\nabla J(\theta)$$ $$v \leftarrow \beta_2 v + (1 - \beta_2)(\nabla J(\theta) \odot \nabla J(\theta))$$ $$\theta \...
0
votes
0answers
14 views

Concatenating Encoder hidden states in LSTM pytorch

I am implementing a seq2seq autoencoder in pytorch: Q1) While it is true that we can keep the encoder as bidirectional, but can we keep the decoder as bidirectional as well(does it make any sense) if ...
0
votes
1answer
12 views

Data preprocessing for time series prediction

I have a dataset that has the following structure ...
1
vote
1answer
24 views

Loaded model predicts well in colab but gives same label and accuracy when downloaded

I have developed a Recurrent Neural Network to perform sentiment analysis on tweets using the Kazanova/sentiment140 dataset in Kaggle. The model looks like this: ...
0
votes
1answer
21 views

Does a CNN think things inside the filter are collocated aka dependent on each other?

I am running a 1D CNN on tabular data. The rows are data that I have are not sequential, that is to say they are not part of a time series or ordered string, which is why I am not using an LSTM. So ...
3
votes
0answers
31 views

N-grams for RNNs

Given a word $w_{n}$ a statistical model such a Markov chain using n-grams predicts the subsequent word $w_{n+1}$. The prediction is by no means random. How is this translated into a neural model? I ...
0
votes
0answers
12 views

Predict all possible dates in time series data

In my problem statement one part is to predict all possible dates(t-n) where t is my current date. I want to process below dataset to predict all possible order dates for next login date(single ...
4
votes
1answer
26 views

Why are character level models considered less effective than word level models?

I have read that character level models need more computation power than word embeddings, and this is one of the major reasons for their less effectiveness, but i got curious because the word ...
2
votes
1answer
31 views

Can bidirectional RNN use variable sequence length?

A bidirectional RNN consists of two RNNs, one for the forward and another for the backward sequential directions, which outcome is concatenated at each time step. Would this configuration restrict the ...
0
votes
1answer
37 views

What is the best way of combining audio and visual data to make predictions?

I am trying to predict the probability of a disease by using audio and images, the audio and the images do not come from the same source. I am thinking of combining the outputs (maybe average them) of ...
0
votes
0answers
18 views

Trying to understand neural network's performance

I am trying to build a RNN for classification and below is the layout of the network ...
0
votes
0answers
17 views

RNN: Multiple inputs per time step with categorical variables

I am trying to a build RNN model to forecast daily sales for several different cities and different product segments (categorical features and multiple inputs for each day) along with numerical ...
0
votes
1answer
69 views

pandas TypeError: 'range' object cannot be interpreted as an integer

I'm following this link for time-series forecasting. While splitting data set to create the data for the uni-variate model this ...
2
votes
0answers
18 views

Predicting parameters of simple configured trajectories using RNN

What I'd like to do: predict orbital elements given an input observation sequence in 2D, that is $$input = X = [position_{t_{0}}, position_{t_{1}},\ ..., position_{T}]$$ $$output = y = parameters\ ...
0
votes
1answer
34 views

How to understand Inconsistent and ambiguous dimensions of matrices used in the Attention layer?

Attention-scoring mechanism seems to be a commonly-used component in various seq2seq models, and I was reading about the original "Location-based Attention" in Bahadanau well-known paper at https://...
0
votes
0answers
19 views

Using RNN to predict future power usage

So for each user in a file I have their average power usage value every hour for 40 consecutive days. I need to predict their power usage during next 10 consecutive days. I am new to RNNs (I have ...
0
votes
1answer
16 views

Why is my LSTM is working best with batch size of 2 and no hidden layers?

I am building an LSTM for price prediction using Keras. I am using Bayesian optimization to find the right hyperparameters. With ...
0
votes
1answer
16 views

Generating Dinosaur names with Tensorflow RNN

I try to adapt "Text generation with an RNN" tutorial to generate new dinosaur names from a list of the existing ones. For training RNN tutorial text is divided into example character sequences of ...
0
votes
0answers
12 views

Inserting input representation at each step of LSTM

In want to train a neural net to generate lyrics based on a provided melody. For that, I have to implement a recurrent neural net (LSTM or GRU) to carry out the task. During each step of the training ...
0
votes
0answers
7 views

How exactly the hidden state works in an RNN ? How to decide on how many past instances to consider?

I am unable to grasp the working of RNN because in different tutorials, it is explained differently. Please correct me as I have considered that: In a ...
1
vote
0answers
12 views

Is it a good idea to disable or strongly regularize in time series deep learning models?

I'm training a recurrent network on a stock price time series. As you can imagine, the price increases with time. I think the importance of the bias decreases as the stock increases, especially since ...
0
votes
2answers
120 views

LSTM Multivariate time series forecasting with multiple inputs for each time step

I want to predict an output variable for the next day, for each of the users in my dataset. I was thinking of using LSTMs for achieving this. The dataset The dataset I am using has multiple inputs ...
0
votes
2answers
20 views

How to Find the Average of the Input Vectors

I want to find the average of input vectors. I tried to use tf.math.reduce_mean, but it went error. If I have to use keras.layers.Average, I have to make a list of the hidden states. Does anyone ...
0
votes
0answers
11 views

Model architecture approaches for event prediction at different timestamps

I would like to model a user event outcome (currently its binary). the data I have is aggregated user activity and static user data. here is an example of what the data looks like for clarity: ...
0
votes
0answers
10 views

AI architecture for time and spacial sequences

I am working on a project where I analyse MEG data. I have 102 channels as a vector and a 2D matrix of the channels (11x14) to show spatial relations - I want to include that in the AI architecture. ...
0
votes
1answer
24 views

Is vanilla RNN suitable for time series prediction?

I read this document: https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/ It was pretty simple, but I don't understand how to use it for predict the next sequence (for example) in trading ...
1
vote
0answers
15 views

Scaling monotonically increasing features between 0 and 1

To keep the test set blind to the neural network algorithm it is generally better to build a scaler based on the training set and then scale the test set on this scaler. I am building an LSTM for ...
0
votes
0answers
23 views

Entity recognition and linkage for reference data

I need to work on a project that deals with recognizing the attributes of references (citations) as Author, Co-Authors, Title of the paper, Publication Venue etc and map them to the real-world ...
0
votes
0answers
14 views

How does stateful LSTM work with keras' batch_size > 1?

Let's say I have one input feature with 10 sequences of length 25 to predict the next value. So keras will receive an input vector of (10, 25, 1). If I use ...
0
votes
0answers
8 views

Suitable model architecture for categorical, sequential data

Suppose you have a dataset, containing the log data of a set of complex devices, e.g. turbofans. For each device, the log consists of a time sequence of categorical events. ...
1
vote
2answers
19 views

Inconsistency on A. Graves' original Connectionist Temporal Classification (CTC) paper?

Here's the link to the paper. For the forward-backward algorithm, they introduce $\alpha_t(s)$ as a definition in eq. (5). Then they give a recursive formula for it (eq. (6)), and its initialization. ...
1
vote
0answers
17 views

LSTM / GRU prediction with hidden state?

I am trying to predict a value based on time series by series of 24 periods (the 25th period) While training I have a validation set with I babysit the training (RMSE) and each epoch, eval the ...
0
votes
0answers
17 views

What's the best small dataset to train a RNN?

I work on a neural network library in C++ and I'm in the process of implementing recurrent neural networks. I run my tests on a GitHub server on different datasets, Iris and Wine for simple unit tests,...
2
votes
2answers
60 views

why we sample when predicting with Recurent Neural Network

I trained a Recurrent Neural Network to predict the next word in a sentence. I trained and now I want to predict, but there is something I am not getting well. I saw it in many tutorials even in the ...
0
votes
0answers
19 views

How to handle a timestep length of 0 in a LSTM model?

So I have a variable input size LSTM model similar to the one from this answer but with one-hot encoded variables. The x_train in my generator is shaped like this: (batch_size, timestep_length, ...
0
votes
0answers
18 views

How do I multiple individual time-series data to train a LSTM model?

I have 100 univariate time-series from individual patients measuring glucose levels. Each time-series is ~20k. I need to train my LSTM model using all of the data. I don't think concatenating all ...
0
votes
0answers
39 views

Regression by PhasedLSTM with a gradient explosion

I found PhasedLSTM inspirational, and used it (PLSTM: Phased LSTM in Keras) to perform the regression (to find the correlation between an input sequence and an output sequence), with Adam optimizer <...
1
vote
0answers
12 views

Creating a RNN dataset and model for understanding context of a sports event

I'm working on a project which requires my code to understand what is happening at any instant in a video feed of a sports match. For example, take a football match. At a given moment, my code focuses ...
1
vote
0answers
19 views

Multivariate LSTM RNN DNN returning multiple features for forecasting a time series in Python

I am using the latest installation of Keras with Python 3.6 on Linux Mint with a NVIDIA (NVDA) 2070 GPU. I am looking up. How to get the return values of my data? How do I use all of the features, and ...
0
votes
0answers
9 views

Truncated Backpropagation Through Time (TBBTT) in Reinforcement Learning

I am currently looking at the OpenAI Five paper from OpenAI. For backpropagation they write: ...

1
2 3 4 5
10