Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

Adapting PyTorch GRU Model for Variable Number of Detected Objects

I'm currently grappling with an issue while working with a PyTorch GRU model in a dynamic scenario where the number of detected objects can vary. The model is specifically designed to process bounding ...
RACHID BEN ABDELMALEK's user avatar
0 votes
0 answers
12 views

Tensorflow RNN - implementing recursive layer

I am dealing with a regression problem, for which I wanted to try to use a recurrent neural network. The general setting is that I have to predict a continuous quantity starting from the evolution, in ...
ChristianC's user avatar
0 votes
2 answers
26 views

Binary classification using RNN not going beyond 50% accuracy

I am trying to find out the reason behind why my RNN network won't go beyond 50% for binary classification. My input data is of the shape: ...
Prabhjot Singh Rai's user avatar
0 votes
1 answer
27 views

Converting a Standard LSTM RNN over to a Transformer Model

I am looking for some advice on converting my existing CNN/LSTM RNN over to a Transformer type model. This regression model takes a sliding window size of 240 rows with 33 features. It aims to ...
Ted Wilmont's user avatar
0 votes
0 answers
30 views

Improving Wake-Word Detection Model Performance: Seeking Advice and Suggestions

I was assigned a task to train a wake-word detection model. Basically, it's a binary sequence classification model on audio samples where it should be 1 if it recognizes the wake word being said (e.g. ...
Ícaro Lorran's user avatar
0 votes
0 answers
15 views

Recurrent neural networks (RNN) and popular recurrence relationships

I am looking for a clean problem to study learning process of a RNN. I am thinking of using some number sequences that appear in mathematics and physics, like Fibonacci's, recurrence relations between ...
D Caden's user avatar
0 votes
0 answers
7 views

Has someone designed a neural network which can select its own activation functions and/or have multiple activation functions in one model?

I'm wonder if there are any papers or implementations where a neural network has multiple activation functions in a single model (and layer), and preferably also where such activation functions ...
BigMistake's user avatar
2 votes
1 answer
113 views

Role of stateful parameter vs shuffle parameter in LSTM keras

I'm trying to make prediction on a multivariate time series using LSTM. I know stateful=True in keras LSTM means state(hidden) of each sequence, in a batch, at index i - is passed to the next batch, ...
the_he_man's user avatar
1 vote
1 answer
61 views

Is timeseries forecasting for the next timeslot with a RNN a "Many-To-One" type application?

you often find applications that divide RNN according to their input and output data into the categories: One-To-One One-To-Many Many-To-One Many-To-Many as you can see e.g. (here https://...
PeterBe's user avatar
  • 81
0 votes
0 answers
28 views

Math Behind Additive Bahdanau Attention

I am new to NLP field and wanted to apply attention model in one of my projects. I have LSTM model to train, and concatenate some external data sources though attention mechanism. The hidden state ...
user154214's user avatar
1 vote
1 answer
50 views

Why my validation loss and accuracy decays over epochs?

Im trying to build 2 simple networks with cleaned dataset for tweets sentiment classification(0/1): one with all dense layers(binary bag of words) another with RNN layer(embedding layer). But it both ...
emily 's user avatar
  • 35
0 votes
0 answers
17 views

Is this problem a time series regression or seq2seq regression or some other type of problem?

I measure sequences of 3 parameters in my system. 2 are independent and the 3rd dependent. Let's call the independent ones $x$ and $y$, and the dependent one $z$. They are each measured once per hour ...
Hitanshu Sachania's user avatar
0 votes
0 answers
15 views

Sending rolling statistics to RNN

I'm curious if anyone has seen cases where sending rolling statistics such as mean, median, min, max, standard deviation, skewness, kurtosis, etc. have been helpful for model accuracy? If so please ...
noNameTed's user avatar
0 votes
0 answers
18 views

Forecasting with exogenous variables

I have the following time-series data: ...
Louis GRIMALDI's user avatar
0 votes
0 answers
16 views

Matrix time-series Forecasting with LSTM

I have the following time-series data: ...
Louis GRIMALDI's user avatar
0 votes
0 answers
18 views

Detecting Anomalies Using LSTMs

I'm studying this article. The authors used a two-step approach to detect anomalies. First, they used an LSTM to learn the normal behavior of the data. Then, they used the dynamic error thresholds to ...
Vahid Shams's user avatar
0 votes
1 answer
50 views

Are there any R packages that support Deep RNNs?

I recently found an interesting paper on what it really means for a recurrent neural network (RNN) to be deep here. Depth can be added in several different ways (state to state, input to state, etc.) ...
noNameTed's user avatar
0 votes
0 answers
38 views

Am I using a network that is too simple for the dataset/task?

I am training an RNN to classify some high-frequency financial data. A very good performance on this data would be an accuracy of >52% or so. I have around 650K training examples and 150K dev set ...
BYZZav's user avatar
  • 103
0 votes
0 answers
17 views

Finding a template RNN for time series analysis

I would like to create a RNN, that uses one (A) or several time series (with the same length, A, B, C...) as an input and creates another time series (Z) as an output from that . Basically all time ...
xlaub's user avatar
  • 1
0 votes
0 answers
16 views

Error in seq2seq translation when passing predicted output to rnn due to input shape not always being the same

I'm working on a language translator and I'm getting an error I'm unsure about. During the decoding process when using argmax on the predicted output I am sometimes getting an RuntimeError ...
bailey.bailey's user avatar
0 votes
0 answers
13 views

Use unclassified texts to improve BERT and RNN GRUN token classification model

I have a training (gold) labelled dataset than consists of 10000 sentences. The task is to create a model that classifies correctly unseen data with B-I-O tags. I have used a BERT and a GRU RNN model. ...
Spyros Triantsfyllou's user avatar
1 vote
0 answers
36 views

ML Modeling Recommendation for Predicting Snake Encounters in Historical Journey Data

I have a dataset consisting of historical journey data where individuals travel from point A to point B. During their journeys, they may encounter varying numbers of animal sightings, including snakes....
Sita's user avatar
  • 11
1 vote
0 answers
41 views

LSTM for classification

I am new to neural networks and I want to use LSTM to classify the on/off state of devices based on power values. In my training dataset, I have power values, device one (0,1), and device 2 (0,1). 0 ...
Zain's user avatar
  • 11
1 vote
1 answer
68 views

Which preprocessing is the correct way to forecast time-series data using LSTM?

I just started to study time-series forecasting using RNN. I have a few months of time series data that was an hour unit. The data is a kind of percentage value of my little experiment and no other ...
orde.r's user avatar
  • 113
0 votes
1 answer
343 views

How to Implement padding and masking sequences for RNN

As an exercise, I'm building a network for binary classification of sequences (whether a sequence belongs to type A or type B). The network consists of an RNN with one LSTM layer, and on top of it an ...
kodkod's user avatar
  • 103
1 vote
2 answers
786 views

RNN with PyTorch - I don't understand the initial parameters

I would like to understand the pyTorch RNN module in detail. There I created a very simple and basic example: ...
Thomas K's user avatar
0 votes
0 answers
10 views

RNN Time Series Footfall - How do I construct this RNN?

I have daily time-series data, which tells me the rain fall & foot fall at a certain shop on that day. Now, I want to predict the foot fall at time $t$, given the previous $2$ observations. As I'm ...
the man's user avatar
  • 101
0 votes
1 answer
29 views

Confusion regarding what constitutes a feature in a LSTM?

I have a Time Series problem, where I am trying to predict a single output at time $t$, $y_t$, given the $2$ previous time steps; $X_{t-2}, X_{t-1}$. Let's just look at one observation for simplicity. ...
the man's user avatar
  • 101
1 vote
0 answers
323 views

Understanding batch size, sequence, sequence length and batch length of a RNN

My Problem I'm struggling with the different definitions of batch size, sequence, sequence length and batch length of a RNN and how to use it in the correct way. First things first - let's clarify the ...
Thomas K's user avatar
5 votes
2 answers
303 views

Modeling uncertainty from known physics

I have an equation given by: $$ \frac{\mathrm{d} s}{\mathrm{d} t}=4a−2s+\lambda(s) $$ where, $a$ is an input constant and $\lambda$ is a non-linear term that depends on $s$. I know that the true ...
user avatar
0 votes
0 answers
19 views

Preparing LSTM Network input from multiple files

I would like to train LSTM Network, which should take 5 files as input and predict the 6th file. Each file contains 810000 data points (precipitation values), and each data point indices the location. ...
karnati shiva's user avatar
0 votes
1 answer
122 views

How many parameter in an RNN?

I came across this question asking about the number of parameters in an RNN layer, from my understanding it is the number of weights and biases, which in this case is five. Can someone confirm this? ...
user149915's user avatar
0 votes
1 answer
438 views

How to make an RNN model in PyTorch that has a custom hidden layer(s) and that is compatible with PackedSequence

I want to make an RNN that has for example more hidden layers or layer normalization. I know that is it possible to make a custom RNN by subclassing nn.module, but with this approach is it not ...
Philip T 2007's user avatar
4 votes
1 answer
157 views

Working with time series data with several times stamps on a dates, and implementing machine learning

I'm trying to implement predictive analytics on a production data. my goal is to predict next downtime, it's reason and issues. My data looks like below; ...
def __init__'s user avatar
0 votes
1 answer
28 views

Why is a RNN inherently better for Time series than normal NN?

Similar to this question but I would like further clarification. I understand that in abstract, RNNs can process inputs recursively and feed some state of memory through the recursion to have a sense ...
Pierre's user avatar
  • 3
0 votes
1 answer
50 views

How is RNN decoder output calculated?

I was trying to read RNN Encoder Decoder paper. RNN (plain RNN i.e. non encoder-decoder RNN) It starts with giving equation for RNN: hidden state in RNN is given as: ... equation (1) where f is a ...
Mahesha999's user avatar
0 votes
0 answers
25 views

which are the best machine learning models for time series data

I have real-time data of a production line that shows when the machine incurs downtime(i.e machine stops functioning). My idea is to predict the next downtime of the machine in well advance. what ...
Satya's user avatar
  • 101
1 vote
1 answer
49 views

Below text-classification model gives accuracy of 0.77 only on one dataset and 0.99 on spam-ham dataset? What should I do to increase with my dataset?

...
rutvi's user avatar
  • 19
2 votes
4 answers
149 views

State-of-the-art LSTM solutions to known datasets

I've seen that many ML datasets have competitions (like imageNet). I've been looking for some kind of competition or state-of-the-art LSTM solutions for The Airline Passengers dataset but all I can ...
Natanael's user avatar
0 votes
0 answers
27 views

Is there a way for CTC to output different types of blanks?

I am using a CTC loss for math handwriting recognition in Tensorflow/Keras. The output consists of a sequence of symbol ids, with a spatial relationship between every pair of consecutive symbols. For ...
Aiden Yun's user avatar
0 votes
0 answers
212 views

RuntimeError: Caught RuntimeError in replica x on device x. Original Traceback: ... RuntimeError: shape '[x, x, x]' is invalid for input of size xxx

I am encountering a RuntimeError with the following message: ...
Youssef Benhachem's user avatar
0 votes
0 answers
40 views

I am creating an multilayer LSTM model from scratch and everything seems to be mathematically correct however the model refuses to learn

I am creating the LSTM with just numpy and plotting the loss with pyplot. I have checked the derivatives again and again however have not found a mistake. The entire code with the main function can be ...
Lanttu1243's user avatar
0 votes
0 answers
19 views

Many To One LSTM - Can I Use the Same Sequence as Input from Previous Timesteps?

I'm new to LSTMs, and I'm trying to do a basic timeseries prediction using stock prices. However, I'm a bit confused as to how the LSTM is supposed to remember outputs from previous timesteps when it ...
Krusty the Clown's user avatar
0 votes
0 answers
18 views

How to derive expression for gradient in BPPT

I have the following problem: I am trying to derive final expressions for error gradients in a simple recurrent neural network (Backpropagation through Time, BPPT). The parameters and state update ...
doktormatte's user avatar
0 votes
0 answers
20 views

Why we need encoder-decoder architectures despite we already have RNN?

Why we need encoder-decoder architectures despite we already have RNN? From Googling, I was just told such architecture is used, in the context of NLP, that it allows: The key benefits of the ...
Student's user avatar
  • 411
0 votes
0 answers
17 views

How to handle dimensionality differences over time or between subjects

Note: This question has in mind tabular data, rather than imaging/NLP. In the situation of collecting data over long periods of time, instruments may change and collect more precise data. This leads ...
user3219765's user avatar
0 votes
1 answer
21 views

Is there such a thing as RNN-LSTM

From the title, I wanna know if there's such a thing as RNN-LSTM. I know that they are their own thing but I've yet to know if there's such a "combination". For context, I was reading a ...
A N D E U S's user avatar
0 votes
1 answer
80 views

How RNN or LSTM delays the input

RNN or LSTM are known to hold the previous timestamp data as "memory" so that short or long range dependencies can be remembered. But in the following simple keras model, where is that delay ...
Sandeep Bhutani's user avatar
1 vote
1 answer
376 views

Why are the hidden states of an RNN initialised every epoch instead of every batch?

Why are the hidden states of RNNs/LSTMs/GRUs generally re-initialised only once an epoch has finished, not once a batch has finished?
postnubilaphoebus's user avatar
0 votes
0 answers
19 views

Is wear and tear calculation on sensor data without labels feasible?

I am currently working on multivariate sensor data from different industrial machines. I was given the task to calculate the wear and tear of different machines. It is expected that the wear and tear ...
Eggplant3654980's user avatar

1
2 3 4 5
15