Questions tagged [rnn]
A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.
744
questions
0
votes
0
answers
2
views
Is there a way for CTC to output different types of blanks?
I am using a CTC loss for math handwriting recognition in Tensorflow/Keras. The output consists of a sequence of symbol ids, with a spatial relationship between every pair of consecutive symbols. For ...
0
votes
0
answers
18
views
RuntimeError: Caught RuntimeError in replica x on device x. Original Traceback: ... RuntimeError: shape '[x, x, x]' is invalid for input of size xxx
I am encountering a RuntimeError with the following message:
...
0
votes
0
answers
9
views
I am creating an multilayer LSTM model from scratch and everything seems to be mathematically correct however the model refuses to learn
I am creating the LSTM with just numpy and plotting the loss with pyplot. I have checked the derivatives again and again however have not found a mistake. The entire code with the main function can be ...
0
votes
0
answers
11
views
Many To One LSTM - Can I Use the Same Sequence as Input from Previous Timesteps?
I'm new to LSTMs, and I'm trying to do a basic timeseries prediction using stock prices. However, I'm a bit confused as to how the LSTM is supposed to remember outputs from previous timesteps when it ...
- 101
0
votes
0
answers
15
views
How to derive expression for gradient in BPPT
I have the following problem: I am trying to derive final expressions for error gradients in a simple recurrent neural network (Backpropagation through Time, BPPT). The parameters and state update ...
0
votes
0
answers
16
views
Why we need encoder-decoder architectures despite we already have RNN?
Why we need encoder-decoder architectures despite we already have RNN? From Googling, I was just told such architecture is used, in the context of NLP, that it allows:
The key benefits of the ...
- 397
0
votes
0
answers
15
views
How to handle dimensionality differences over time or between subjects
Note: This question has in mind tabular data, rather than imaging/NLP.
In the situation of collecting data over long periods of time, instruments may change and collect more precise data. This leads ...
0
votes
1
answer
17
views
Is there such a thing as RNN-LSTM
From the title, I wanna know if there's such a thing as RNN-LSTM. I know that they are their own thing but I've yet to know if there's such a "combination".
For context, I was reading a ...
0
votes
1
answer
15
views
How RNN or LSTM delays the input
RNN or LSTM are known to hold the previous timestamp data as "memory" so that short or long range dependencies can be remembered.
But in the following simple keras model, where is that delay ...
- 864
0
votes
1
answer
27
views
Why are the hidden states of an RNN initialised every epoch instead of every batch?
Why are the hidden states of RNNs/LSTMs/GRUs generally re-initialised only once an epoch has finished, not once a batch has finished?
0
votes
0
answers
18
views
Is wear and tear calculation on sensor data without labels feasible?
I am currently working on multivariate sensor data from different industrial machines. I was given the task to calculate the wear and tear of different machines.
It is expected that the wear and tear ...
0
votes
0
answers
7
views
Help with transitioning an existing DQN into a DRQN
Hi Data Science Stack Exchange community,
To preface this post, please let me know if I need to clarify any details to receive help and/or guidance. I am new to posting on Data Science Stack Exchange ...
- 1
1
vote
0
answers
14
views
Epochs for new batch when online training?
I am online training a RNN with fixed batch size k on a time series. Initially I train my model with n batches and a number of e epochs. When a new batch n+1 is available, I would like to update the ...
- 11
1
vote
1
answer
41
views
Using the whole GloVe pre-trained embedding matrix or minimize the matrix based on the number of words in vocabulary
I have created a neural network for sentiment analysis using bidirectional LSTM layers and pre-trained GloVe embeddings.
During the training I noticed that the ...
- 111
0
votes
0
answers
33
views
Why are GRU layer dimensions incompatible using ragged tensor input?
I am attempting to create a sequential model in Keras that accepts a 3-dimensional ragged tensor as an input, creates an embedding, and feeds into a GRU layer. While I can get the model to accept the ...
- 1
0
votes
0
answers
20
views
Keras LSTM's: why do I get the best results with just 1 timestep input data?
I've created a stacked LSTM model to predict the price of Bitcoin for each next timestep ( day ) based on the historic values; say, the values of the last n ...
- 101
2
votes
2
answers
333
views
How to arrange multiple multivariate time series of different length before passing it to Keras LSTM layer
I have a number of multivariate time series that are produced by the same kind of process but:
are of significantly different lengths;
each time series is an independent instance, and the ...
- 182
0
votes
0
answers
7
views
What does it mean to "condition' a net's output?
Graves talks about conditioning the predictions of a net based on inputs. What does that mean, and how is it done?
- 101
0
votes
0
answers
45
views
0
votes
0
answers
37
views
Why are LSTMs not very good in extrapolating time-series?
I was trying to train an LSTM based recurrent-neural-network to extrapolate a simple time-series. The time-series I am using a simple superposition of sinusoidal series of different frequencies.
...
- 1
0
votes
0
answers
28
views
Supervided anomaly detection for time series sensor data with LSTM
I have a supervised anomaly detection problem which involves time series sensor data.
The structure of data is as follows:
I have several industrial machines (say 100).
Each machine is equipped with ...
- 43
0
votes
0
answers
44
views
Difference between batch_size in TimeSeriesGenerator and model.fit batch_size
im wondering if there is a difference between the batch_size set in the TimeSeriesGenerator and the batch_size in the model_fit. I create some RNN Forecasts for timeseries.
...
- 11
0
votes
1
answer
25
views
How is loss calculated in truncated BPTT, for a many to one problem?
In many resources I refered to such as Justin Johnson's Lecture 12 on RNN, truncated BPTT is explained as the process of feedforward and backpropagate for smaller chunks of the sequence. These ...
- 13
0
votes
2
answers
43
views
RNN/LSTM with multiple targets and varying sequence
I have a dataset containing readings from different sensors. Each sensor can provide distance and signal strength but those data is not always available. At each time interval only three sensors can ...
0
votes
0
answers
539
views
can't convert lists to tensor in tensorflow python
I'm relatively new to ML and data science and I'm using tensorflow and keras to do a NLP project.
I have about 18000 emails, in my code I convert each word in every email to a vector of shape (1,50) ...
- 1
0
votes
0
answers
29
views
Input size vs hidden state in RNNs
Im using PyTorch to implement RNNs on univariate time series data. This is the documentation for the RNN class: link
I think I'm understanding the math behind an RNN cell. But I have an specific ...
- 101
1
vote
1
answer
23
views
Should I annotate additional information besides the categories I already need in a text?
I have a dataset with bank transfer reasons. They vary a lot because humans wrote them.
From the reasons that are linked to invoice payments I need to extract several things:
invoice number(s)
IBAN
...
- 155
1
vote
0
answers
280
views
LSTM Forecast timeseries with Hyperparameter Tuner (Random Search) from Keras
I want to predict a timeseries with a LSTM Model.
I try to use the Tuner from Keras to find the best hyperparameters.
data_example:
date
value
2022-01-02
600
2022-01-03
640
2022-01-04
605
...
...
...
- 11
0
votes
0
answers
16
views
What do results like these imply in a LSTM classification problem?
I am training a LSTM network to learn from multiple time series, and the output from the network should be binary (or equivalently a probability score between [0, 1]...
- 1
0
votes
0
answers
21
views
Predicting time series
I have a very large dataset (about a year of driving) which contains the following features:
datetime with 1 second resolution - speed of car - GPS coordinates of the car in each time-step - average ...
0
votes
0
answers
13
views
Degree of freedom for NLP DL models
How degree of freedom can be estimated for NLP use cases where put is high dimensional vector (let us say word2vec used and dim size is 500) say and RNN or 1D CNN is used for modeling?
- 101
0
votes
1
answer
26
views
RNN for continuous, real-time learning without pre-training
I am learning ML and I'm trying to solve this problem
Create a rock paper scissors game where the AI is able to beat the player more than 50% of the time.
My initial intuition was to use an RNN with ...
0
votes
1
answer
33
views
What counts as a token for bpemb's encode_ids_with_eos()
I have probelms understanding bpemb's encode_ids_with_eos() or similar.
When I run the following code i get none-word like segmentations (rather syllalbus based or ...
- 125
0
votes
1
answer
70
views
Can MLP model sequential data?
When modeling sequential data, RNNs are introduced as an improvement of MLP as they can model the time dependency between the inputs. It is said that feeding the last N data points in the sequence to ...
0
votes
1
answer
101
views
pytorchs LSTMs use of 'bias' and 'weight' strings
Hi I am new to RNN and have come across this the following implementation of Pytorchs LSTM, but I cant understand how (or why) the 'bias' and ...
- 125
0
votes
0
answers
90
views
How to use multiple parallel inputs for time series forecasting -- Pytorch
I'm currently working with the ECG recordings of several patients. I have the amplitude of the ECG for around 48 patients over the span of one hour, and I want to be able to forecast their future ECG ...
0
votes
0
answers
21
views
Surrogate model for [parameter vector] to [time series]
Say I have a model $M$ that takes in a parameter vector $\beta$, and produces a (numerical) time series. This could be a complicated model (e.g. a bespoke enzyme reaction model), or something simple ...
- 13
0
votes
0
answers
18
views
Contextual Embeddings LSTM DOUBT
I have a simple doubt.
When we use Word2Vec, Its obviously a non contextual embedding because every word has a same representation. When I pass it to my LSTM, We say the hidden states are the ...
- 109
0
votes
0
answers
41
views
Validation loss stays bouncing while training loss converges immediately
I'm using bi-GRU to try solving a bi-classification problem.
What I have observed is that no matter how much dropout(from 0 to 0.6) and layer-norm I added, the training process shows similar situation:...
- 1
0
votes
0
answers
27
views
Building a machine learning model to predict variables measured after the end of the crop based on environmental data
I have dataset (20 samples) containing timeseries on temperature, humidity etc (a total of 6 variables). Each timeseries is 5 complete days, which is 24 * 5 = 120 values.
So dataset has hourly values ...
- 1
0
votes
0
answers
49
views
DRQN Cartpole v-1 with decreasing reward
I'm trying to use RNN instead of feed-forward NN for the Cartpole-v1 problem but I cannot figure out why the reward seems to be decreasing. I thought the problem might be due to the fact that the ...
- 1
0
votes
0
answers
10
views
What is the procedure for data preprocessing for time-dependent LSTM classifier?
I attempt a beginner level LSTM classification task with a time-series numerical data, but my task is finding changes in features over time (in which those changes describe the outcome or the classes),...
1
vote
1
answer
518
views
Difference Between Attention and Fully Connected Layers in Deep Learning
There have been several papers in the last few years on the so-called "Attention" mechanism in deep learning (e.g. 1 2). The concept seems to be that we want the neural network to focus on ...
- 726
1
vote
1
answer
55
views
Multivariate timeseries classification for each group in a dataset
Let's say, I have the following dataset:
...
- 21
0
votes
0
answers
4
views
How To Shuffle Long-Short-Term-Memory Or Gated-Recurrent-Unit Layer Cells Operation?
As you know these types of layers operate side-to-side, and although could have been implemented in Bidirectional layer to operate on both forward and backward directions.
But is it possible to change ...
- 121
0
votes
0
answers
20
views
Validation error approaches the same value for many hyperparameters
I am using kerastuner to explore the parameter space of my RNN. The validation MSE for each model seems to follow the same trend: completely level at ~0.5, a major drop around the third epoch, then ...
1
vote
1
answer
69
views
Why do RNN text generation models treat word prediction as a classification task?
In many of the sources I have found regarding text generation with word-based RNN models (LSTM or GRU), the model is trained to perform a classification task across the vocabulary (such as with ...
- 113
0
votes
0
answers
18
views
Temporal rows selection for Recurrent Neural Networks
I have a time serie $x_{1},...,x_{n}$ with a temporal step $\Delta = date(x_{i+1}) - date(x_{i}) = (i+1) - (i)= 1 \ day $. For each $i \in [\![ 1,n ]\!] $, I know that the value of $x_{i}$ depends ...
- 1
0
votes
1
answer
19
views
RNN basic doubt
Suppose if I have 2 sentences:
"My name is Alex"
"Alex is my name"
If I am using a RNN, After processing both the sentences, Will the final output vector be the same?
Because RNN ...
- 109
0
votes
0
answers
21
views
Machine Learning Model to Translate an Input Time Series to a Target Time Series?
I want to train a machine learning model to translate input time series signals into target (ground truth) time series signals. I have thousands of input-target training pairs similar to the ones ...
- 1