Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
1 vote
1 answer
9 views

Which preprocessing is the correct way to forecast time-series data using LSTM?

I just started to study time-series forecasting using RNN. I have a few months of time series data that was an hour unit. The data is a kind of percentage value of my little experiment and no other ...
orde.r's user avatar
  • 111
0 votes
1 answer
23 views

How to Implement padding and masking sequences for RNN

As an exercise, I'm building a network for binary classification of sequences (whether a sequence belongs to type A or type B). The network consists of an RNN with one LSTM layer, and on top of it an ...
kodkod's user avatar
  • 103
0 votes
2 answers
38 views

RNN with PyTorch - I don't understand the initial parameters

I would like to understand the pyTorch RNN module in detail. There I created a very simple and basic example: ...
Thomas K's user avatar
0 votes
0 answers
5 views

RNN Time Series Footfall - How do I construct this RNN?

I have daily time-series data, which tells me the rain fall & foot fall at a certain shop on that day. Now, I want to predict the foot fall at time $t$, given the previous $2$ observations. As I'm ...
the man's user avatar
  • 101
0 votes
1 answer
23 views

Confusion regarding what constitutes a feature in a LSTM?

I have a Time Series problem, where I am trying to predict a single output at time $t$, $y_t$, given the $2$ previous time steps; $X_{t-2}, X_{t-1}$. Let's just look at one observation for simplicity. ...
the man's user avatar
  • 101
1 vote
1 answer
27 views

Understanding batch size, sequence, sequence length and batch length of a RNN

My Problem I'm struggling with the different definitions of batch size, sequence, sequence length and batch length of a RNN and how to use it in the correct way. First things first - let's clarify the ...
Thomas K's user avatar
4 votes
2 answers
250 views

Modeling uncertainty from known physics

I have an equation given by: $$ \frac{\mathrm{d} s}{\mathrm{d} t}=4a−2s+\lambda(s) $$ where, $a$ is an input constant and $\lambda$ is a non-linear term that depends on $s$. I know that the true ...
nop nop's user avatar
  • 41
0 votes
0 answers
11 views

Preparing LSTM Network input from multiple files

I would like to train LSTM Network, which should take 5 files as input and predict the 6th file. Each file contains 810000 data points (precipitation values), and each data point indices the location. ...
karnati shiva's user avatar
0 votes
1 answer
43 views

How many parameter in an RNN?

I came across this question asking about the number of parameters in an RNN layer, from my understanding it is the number of weights and biases, which in this case is five. Can someone confirm this? ...
user149915's user avatar
0 votes
1 answer
32 views

How to make an RNN model in PyTorch that has a custom hidden layer(s) and that is compatible with PackedSequence

I want to make an RNN that has for example more hidden layers or layer normalization. I know that is it possible to make a custom RNN by subclassing nn.module, but with this approach is it not ...
Philip T 2007's user avatar
4 votes
1 answer
37 views

Working with time series data with several times stamps on a dates, and implementing machine learning

I'm trying to implement predictive analytics on a production data. my goal is to predict next downtime, it's reason and issues. My data looks like below; ...
def __init__'s user avatar
0 votes
1 answer
25 views

Why is a RNN inherently better for Time series than normal NN?

Similar to this question but I would like further clarification. I understand that in abstract, RNNs can process inputs recursively and feed some state of memory through the recursion to have a sense ...
Pierre's user avatar
  • 3
0 votes
1 answer
19 views

How is RNN decoder output calculated?

I was trying to read RNN Encoder Decoder paper. RNN (plain RNN i.e. non encoder-decoder RNN) It starts with giving equation for RNN: hidden state in RNN is given as: ... equation (1) where f is a ...
Mahesha999's user avatar
0 votes
0 answers
15 views

which are the best machine learning models for time series data

I have real-time data of a production line that shows when the machine incurs downtime(i.e machine stops functioning). My idea is to predict the next downtime of the machine in well advance. what ...
Satya's user avatar
  • 101
1 vote
1 answer
44 views

Below text-classification model gives accuracy of 0.77 only on one dataset and 0.99 on spam-ham dataset? What should I do to increase with my dataset?

...
rutvi's user avatar
  • 19
2 votes
4 answers
100 views

State-of-the-art LSTM solutions to known datasets

I've seen that many ML datasets have competitions (like imageNet). I've been looking for some kind of competition or state-of-the-art LSTM solutions for The Airline Passengers dataset but all I can ...
Natanael's user avatar
0 votes
0 answers
11 views

Is there a way for CTC to output different types of blanks?

I am using a CTC loss for math handwriting recognition in Tensorflow/Keras. The output consists of a sequence of symbol ids, with a spatial relationship between every pair of consecutive symbols. For ...
Aiden Yun's user avatar
0 votes
0 answers
134 views

RuntimeError: Caught RuntimeError in replica x on device x. Original Traceback: ... RuntimeError: shape '[x, x, x]' is invalid for input of size xxx

I am encountering a RuntimeError with the following message: ...
Youssef Benhachem's user avatar
0 votes
0 answers
19 views

I am creating an multilayer LSTM model from scratch and everything seems to be mathematically correct however the model refuses to learn

I am creating the LSTM with just numpy and plotting the loss with pyplot. I have checked the derivatives again and again however have not found a mistake. The entire code with the main function can be ...
Lanttu1243's user avatar
0 votes
0 answers
13 views

Many To One LSTM - Can I Use the Same Sequence as Input from Previous Timesteps?

I'm new to LSTMs, and I'm trying to do a basic timeseries prediction using stock prices. However, I'm a bit confused as to how the LSTM is supposed to remember outputs from previous timesteps when it ...
Krusty the Clown's user avatar
0 votes
0 answers
15 views

How to derive expression for gradient in BPPT

I have the following problem: I am trying to derive final expressions for error gradients in a simple recurrent neural network (Backpropagation through Time, BPPT). The parameters and state update ...
doktormatte's user avatar
0 votes
0 answers
17 views

Why we need encoder-decoder architectures despite we already have RNN?

Why we need encoder-decoder architectures despite we already have RNN? From Googling, I was just told such architecture is used, in the context of NLP, that it allows: The key benefits of the ...
Student's user avatar
  • 409
0 votes
0 answers
16 views

How to handle dimensionality differences over time or between subjects

Note: This question has in mind tabular data, rather than imaging/NLP. In the situation of collecting data over long periods of time, instruments may change and collect more precise data. This leads ...
user3219765's user avatar
0 votes
1 answer
20 views

Is there such a thing as RNN-LSTM

From the title, I wanna know if there's such a thing as RNN-LSTM. I know that they are their own thing but I've yet to know if there's such a "combination". For context, I was reading a ...
A N D E U S's user avatar
0 votes
1 answer
52 views

How RNN or LSTM delays the input

RNN or LSTM are known to hold the previous timestamp data as "memory" so that short or long range dependencies can be remembered. But in the following simple keras model, where is that delay ...
Sandeep Bhutani's user avatar
0 votes
1 answer
106 views

Why are the hidden states of an RNN initialised every epoch instead of every batch?

Why are the hidden states of RNNs/LSTMs/GRUs generally re-initialised only once an epoch has finished, not once a batch has finished?
postnubilaphoebus's user avatar
0 votes
0 answers
18 views

Is wear and tear calculation on sensor data without labels feasible?

I am currently working on multivariate sensor data from different industrial machines. I was given the task to calculate the wear and tear of different machines. It is expected that the wear and tear ...
Eggplant3654980's user avatar
0 votes
0 answers
15 views

Help with transitioning an existing DQN into a DRQN

Hi Data Science Stack Exchange community, To preface this post, please let me know if I need to clarify any details to receive help and/or guidance. I am new to posting on Data Science Stack Exchange ...
Jeremy's user avatar
  • 1
1 vote
0 answers
15 views

Epochs for new batch when online training?

I am online training a RNN with fixed batch size k on a time series. Initially I train my model with n batches and a number of e epochs. When a new batch n+1 is available, I would like to update the ...
Marx's user avatar
  • 11
1 vote
1 answer
103 views

Using the whole GloVe pre-trained embedding matrix or minimize the matrix based on the number of words in vocabulary

I have created a neural network for sentiment analysis using bidirectional LSTM layers and pre-trained GloVe embeddings. During the training I noticed that the ...
NikSp's user avatar
  • 111
0 votes
0 answers
72 views

Why are GRU layer dimensions incompatible using ragged tensor input?

I am attempting to create a sequential model in Keras that accepts a 3-dimensional ragged tensor as an input, creates an embedding, and feeds into a GRU layer. While I can get the model to accept the ...
saganaga's user avatar
0 votes
0 answers
56 views

Keras LSTM's: why do I get the best results with just 1 timestep input data?

I've created a stacked LSTM model to predict the price of Bitcoin for each next timestep ( day ) based on the historic values; say, the values of the last n ...
user2999349's user avatar
2 votes
2 answers
766 views

How to arrange multiple multivariate time series of different length before passing it to Keras LSTM layer

I have a number of multivariate time series that are produced by the same kind of process but: are of significantly different lengths; each time series is an independent instance, and the ...
Xaume's user avatar
  • 182
0 votes
0 answers
7 views

What does it mean to "condition' a net's output?

Graves talks about conditioning the predictions of a net based on inputs. What does that mean, and how is it done?
étale-cohomology's user avatar
0 votes
0 answers
72 views

Understanding stacked LSTM architecture

Consider the following RNN architecture : ...
AAA's user avatar
  • 45
0 votes
0 answers
73 views

Why are LSTMs not very good in extrapolating time-series?

I was trying to train an LSTM based recurrent-neural-network to extrapolate a simple time-series. The time-series I am using a simple superposition of sinusoidal series of different frequencies. ...
pafloxy's user avatar
0 votes
0 answers
57 views

Supervided anomaly detection for time series sensor data with LSTM

I have a supervised anomaly detection problem which involves time series sensor data. The structure of data is as follows: I have several industrial machines (say 100). Each machine is equipped with ...
Farzad's user avatar
  • 43
0 votes
1 answer
86 views

Difference between batch_size in TimeSeriesGenerator and model.fit batch_size

im wondering if there is a difference between the batch_size set in the TimeSeriesGenerator and the batch_size in the model_fit. I create some RNN Forecasts for timeseries. ...
prgtttt's user avatar
  • 11
0 votes
1 answer
31 views

How is loss calculated in truncated BPTT, for a many to one problem?

In many resources I refered to such as Justin Johnson's Lecture 12 on RNN, truncated BPTT is explained as the process of feedforward and backpropagate for smaller chunks of the sequence. These ...
butwhy's user avatar
  • 13
0 votes
2 answers
66 views

RNN/LSTM with multiple targets and varying sequence

I have a dataset containing readings from different sensors. Each sensor can provide distance and signal strength but those data is not always available. At each time interval only three sensors can ...
Geng Wang's user avatar
0 votes
0 answers
765 views

can't convert lists to tensor in tensorflow python

I'm relatively new to ML and data science and I'm using tensorflow and keras to do a NLP project. I have about 18000 emails, in my code I convert each word in every email to a vector of shape (1,50) ...
infinite's user avatar
0 votes
0 answers
32 views

Input size vs hidden state in RNNs

Im using PyTorch to implement RNNs on univariate time series data. This is the documentation for the RNN class: link I think I'm understanding the math behind an RNN cell. But I have an specific ...
RLC's user avatar
  • 101
1 vote
1 answer
26 views

Should I annotate additional information besides the categories I already need in a text?

I have a dataset with bank transfer reasons. They vary a lot because humans wrote them. From the reasons that are linked to invoice payments I need to extract several things: invoice number(s) IBAN ...
Yana's user avatar
  • 165
1 vote
0 answers
399 views

LSTM Forecast timeseries with Hyperparameter Tuner (Random Search) from Keras

I want to predict a timeseries with a LSTM Model. I try to use the Tuner from Keras to find the best hyperparameters. data_example: date value 2022-01-02 600 2022-01-03 640 2022-01-04 605 ... ... ...
prgtttt's user avatar
  • 11
0 votes
0 answers
19 views

What do results like these imply in a LSTM classification problem?

I am training a LSTM network to learn from multiple time series, and the output from the network should be binary (or equivalently a probability score between [0, 1]...
NicTam's user avatar
  • 1
0 votes
0 answers
22 views

Predicting time series

I have a very large dataset (about a year of driving) which contains the following features: datetime with 1 second resolution - speed of car - GPS coordinates of the car in each time-step - average ...
Jonathan Chris's user avatar
0 votes
0 answers
14 views

Degree of freedom for NLP DL models

How degree of freedom can be estimated for NLP use cases where put is high dimensional vector (let us say word2vec used and dim size is 500) say and RNN or 1D CNN is used for modeling?
user140192's user avatar
0 votes
1 answer
28 views

RNN for continuous, real-time learning without pre-training

I am learning ML and I'm trying to solve this problem Create a rock paper scissors game where the AI is able to beat the player more than 50% of the time. My initial intuition was to use an RNN with ...
FrenchMajesty's user avatar
0 votes
1 answer
43 views

What counts as a token for bpemb's encode_ids_with_eos()

I have probelms understanding bpemb's encode_ids_with_eos() or similar. When I run the following code i get none-word like segmentations (rather syllalbus based or ...
Piskator's user avatar
  • 125
0 votes
1 answer
121 views

Can MLP model sequential data?

When modeling sequential data, RNNs are introduced as an improvement of MLP as they can model the time dependency between the inputs. It is said that feeding the last N data points in the sequence to ...
Lukas Petersson's user avatar

1
2 3 4 5
15