Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

multivariate biLSTM text classification

I have made a model that takes vacancy data and classifies it with (bi)LSTM. The variables of the initial dataset are: positiontitle ...
0
votes
1answer
10 views

Normalization of possibly not fully representative data

I am trying to train a classification RNN model on a sequence of table medical data, but I stuck with the normalization problem. I realized that I cannot simply use MinMaxScaler, because of 3 problems:...
0
votes
0answers
13 views

Encoding entities with features of continuous values

Given a set of entities, I would like to predict the next in the sequence; for this purpose, I would like to use RNN. However, my first challenge is how to model the entities. A possible input ...
1
vote
0answers
21 views

One Year Ahead Forecasting with Unevenly Spaced Time Series

I have many products in my warehouses which can be "demanded" any day by my different clients. I want to forecast how many of each item will be demanded for the whole next year. Naturally, ...
2
votes
1answer
25 views

How to implement sequence to sequence models?

I have a dataset with patient demographics, diagnosis history, hospital visit dates, drugs consumed etc. All these events have time stamp information (except static info like demographics such gender, ...
0
votes
0answers
11 views

Keras RNN(LSTM) Loss Not Decreasing

I'm running an LSTM model to classify astrophysics time series data. I am attempting to classify each time series as either a black hole (1) or not a black hole (0). When I run my RNN, the loss is not ...
1
vote
1answer
24 views

Can RNN be replaced with non-recurrent classifier for Sequence Classification problem?

Setup: We have sequence of events that are not evenly spaced (not a time series). Length of the sequence is constant. Goal: Predict class of the event that is most probable to follow this sequence. ...
0
votes
0answers
12 views

Neural Networks with a list of undetermined length as input

I have a list of strings and for each input (each list), there is a target that is again a string. The idea is to let the network learn to generalize from the inputs. Here is one example: ...
0
votes
0answers
29 views

Modelling feature interactions in LSTM network

I created a binary model from 28 features, each sequence is 10 samples long. I tried these two models: ...
0
votes
0answers
8 views

Time series forecasting when one of the series is known

I have a problem where there are two time series $\{x_t\}_{t \geq 1}$ and $\{z_t\}_{t \geq 1}$. These two time series are correlated for fixed time instant but uncorrelated with each other across time....
1
vote
1answer
31 views

LSTM for Stock Return Prediction

I am writing my masters thesis and am using LSTMs for daily stock return prediction. So far I am only predicting numerical values but will soon explore a classification style problem and predict ...
2
votes
1answer
31 views

Recurrent Neural Network (RNN) Vanishing gradient problem - Why does it affect earlier timesteps more?

I understand the concept of backpropagation in standard neural networks and backpropagation through time with RNNs, why this causes exponentially smaller gradients at earlier time steps and most of ...
0
votes
0answers
17 views

Trying to extend this code to include additional feature volume (in addition to adj close) RNN to predict adj close

I read this article on medium https://medium.com/swlh/a-technical-guide-on-rnn-lstm-gru-for-stock-price-prediction-bce2f7f30346 prep ...
0
votes
0answers
24 views

How to Classify Game Stages Based on Bitrate Time Series Data

I need suggestions for my project and would be glad if you would give me a hand. I have a dataset of frames obtained from the old-school game DOOM. Each frame in the dataset has the following columns: ...
0
votes
1answer
21 views

How to improve LSTM accuracy on multiclass text classification?

So, I'm trying to build a LSTM model to classify multiclass text label. The goal is to make a prediction about user rating (1, 2, 3, 4, 5) based on their review. My hyperparameter is like this: ...
0
votes
0answers
28 views

Pytorch: how to pass the hidden state between the samples in LSTM?

I am trying to boost the performance of a object detection task with sequential information, using ConvLSTM. A typical ConvLSTM model takes a 5D tensor with shape ...
1
vote
0answers
12 views

Can Batch Normalization replace tanh in RNN?

Question Can Batch Normalization (BN) be inserted in RNN after $x_t@W_{xh}$, and after $h_{t-1}@W_{hh}$ to remove $f=tanh$ and bias $b_h$? If possible, will this ...
2
votes
0answers
85 views

Elman RNN with keras

I have to perform multi-step multivariate forecasting of time series, using keras. I found an example where LSTM is used. I could modify that example replacing LSTM with SimpleRNN. Now I would like to ...
1
vote
0answers
27 views

Why don't we set initial hidden states in RNN to random small numbers like we do to the weights?

I'm following a couple of tutorials on RNNs and the instructor said that we should always set the initial hidden state in our RNNs to a tensor of all zeros and I couldn't really understand why. Even ...
0
votes
0answers
12 views

speech emotion recongtion cnn or rnn?

I want to build a speech emotion classifier and I labeled my data into 3 emotions {negative, neutral, positive} the speech files I have are different of length, thus my audio features (mfcc,zcr, etc.) ...
0
votes
0answers
9 views

Pytorch RNN with no nonlinearity

Is it possible to implement an RNN layer with no nonlinearity in Pytorch like in Keras where one can set the activation to linear? By removing the nonlinearlity, I want to implement a first-order ...
1
vote
1answer
24 views

Do Recurrent Neural Networks assume stationarity or just a general kind of sequential dependence?

Just when I thought I had convinced myself that RNNs make no other assumption about a sequence other than that there are dependencies between the inputs and that (in the case of monodirectional RNNs) ...
0
votes
0answers
10 views

PyTorch LSTM with varying time steps

Is it possible to create an LSTM in PyTorch where the time steps are varying? For example, heights where measurements are taken at various times. The data might look like this: Person id Inches tall ...
0
votes
0answers
14 views

Treating Null Degrees of an Angle

I have a dataset that measures the flight details of objects, based on what action was performed. It looks similar to below: ...
1
vote
0answers
13 views

How can I convert my predictions to text after predicting using RNN?

I'm building PoS tagger for our language. I give tokens to the words and tags using Tokenizer(). Functions for word and tag are different. ...
1
vote
0answers
24 views

Padding before or after truncation for stateful LSTM in Keras

I am training an LSTM on a dataset with variable timesteps (between 10 and 6000). Using the truncated backpropagation through time (TBPTT) technique, I am truncating the sequences to windows of 128 ...
0
votes
0answers
9 views

RNN predict across new timeseries

For simplicity, I assume that the following timeseries $X_{A}$ and $Y_{A}$ are univariate. I am familiar with training RNN networks, such as LSTM, on a given timeseries $X_{A} = (X_{A0}, X_{A1}, X_{A2}...
-1
votes
2answers
21 views

Optimisation of neural networks

Do neural networks get optimized by trial and error, by data scientists, or is there some way of optimizing values through accurate mathematical equations?
0
votes
0answers
30 views

Output of a network to it's input, keras

I'm trying to create a neural network in keras for time series forecast. I've build a concept, and now I'm not quite sure if it is possible to implement using keras. I have a potentially complicated ...
0
votes
0answers
11 views

What is the encoders job and how does it do that LSTM

I am studying LSTM-RNNs and ran into a problem! I am not sure what exactly the Encoder's job is, or better: how he does it? I know that the Encoder is transforming the input in a way, that the Decoder ...
0
votes
1answer
30 views

Tricky stacking models in keras

I'm trying to write a model with keras, that is built as shown below: ...
0
votes
1answer
30 views

What is the input of LSTM network?

Hello I am trying to understand LSTMs but have a few problems: What is the input? Since LSTM is seq2seq I would think it is a sequence of words, but in a Codecademy lesson is mentioned that each ...
1
vote
0answers
8 views

why would you mask out padded activations from the training loss?

I've followed taming-lstm for training a LSTM model on a NLP task in batches with various sentence lengths. One of his main points is: Trick 3: Mask out network outputs we don’t want to consider in ...
0
votes
1answer
17 views

Difference between zero-padding and character-padding in Recurrent Neural Networks

For RNN's to work efficiently we vectorize the problem which results in an input matrix of shape (m, max_seq_len) where m is the number of examples, e.g. ...
1
vote
1answer
36 views

Timeseries LSTM: does test data need to come after training data?

I have one single, very long time series. I want to train an LSTM to distinguish between two behaviours (A or B) at every timestep (sequence-to-sequence). Because the time series is very long, I plan ...
0
votes
0answers
27 views

Making Keras RNN to proceed input sequence step by step

I'm currently trying to create a neural network for playing Tetris. I'm using evolutionary algorythms for it's learning, so the behavior that I need to get from the neural network is the following: ...
0
votes
1answer
43 views

Regression with LSTM network: use multiple time series as input

I've spent a few days on this and am starting to think I'm missing the obvious solution as this doesn't seem like a very uncommon problem. As an example dataset: I have 100 measurements with each a ...
0
votes
0answers
14 views

Regarding the application of RNN on new data

Suppose the two univariate time series $X_{1,T}=(x_1, x_2, ..., x_T)$ and $Y_{1,T}=(y_1, y_2, ..., y_T)$. The next step would be to train an RNN or LSTM with input $X_{1,T}$ and output $Y_{1,T}$, in ...
0
votes
0answers
19 views

Derivation of HiddenState wrt Output of LSTM

I'm busy trying to understand the math behind LSTM RNN's. In most of the math tutorials that I've found the derivations (Backpropagation) don't consider a dense layer before the output, instead they ...
0
votes
0answers
20 views

Passing data to RNN with Sliding window approach

I having hard time with LSTM's and RNN so my apologies if this question sounds like a very basic question. I would appreciate if you can help in any way. I am trying to train my RNN with LSTMs, but I ...
0
votes
0answers
62 views

How to implement a Multivariate multi-site application in LSTM?

I am trying to make a multivariate multi-site classification LSTM model using Keras. I have followed this tutorial from Jason Brownlee: https://machinelearningmastery.com/multivariate-time-series-...
0
votes
0answers
20 views

How to improve a LSTM self-attention model given the absence of overfitting

I am doing a binary classification on time series data. Class 0 is a single class but class 1 is actually a combination of 7 different classes. My objective to classify class 0 from other classes. The ...
0
votes
0answers
17 views

Share the experiences of regularization of a LSTM model

There are five parameters from an LSTM layer for regularization if I am correct. To deal with overfitting, I would start with reducing the layers reducing the hidden units Applying dropout or ...
1
vote
1answer
33 views

Number of parameters in Simple RNNs

Please, I am stuck, I can not understand the number of parameters of a simple RNN, here the example and the model summary. the example is simple: ...
0
votes
0answers
8 views

Series Through a GPU's Window To, For Each Item, Output a Prediction and Retrain?

Perhaps I'm missing something obvious but I've not run across a Keras or PyTorch example of online training and series prediction loop implemented on a GPU with these (seemingly obvious) ...
1
vote
0answers
14 views

Advice on using Recurrent Neural Networks for non-time series dataset

I'm testing different machine learning algorithms for predicting week-to-week fantasy football scores for individual players. For those who don't know, fantasy football is a game in which players pick ...
0
votes
0answers
12 views

RNN Layer with multiple independent sequences for same label

I have a question about the use of longitudinal data (reoccurring sequences of data). Let's imagine we have multiple independent longitudinal features (e.g. visits to doctor and purchases in online ...
0
votes
0answers
20 views

How far into the future can I forecast a time-series with an LSTM and strongly seasonal data

I am working on a Sequence-to-Sequence + Attention model for some time-series data. Now I have a really long time series, basically 40 years of daily observations for multiple sensors. The data itself ...
0
votes
0answers
41 views

ML Algorithm or approach to solve timeseries selection

I am fairly new to ML and I'm working on a problem, but not sure which algorithm to choose. The dataset contains a set of incremental time-series events, in consistent and set intervals, with a list ...
1
vote
0answers
60 views

Concatenation of CNN and LSTM to model time of a series of images

I have collected a dataset consisting of around 30'000 heat maps of 80 users. The heat maps represent typing behavior on a keyboard and are just images with a resolution of ...

1
2 3 4 5
14