Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

Filter by
Sorted by
Tagged with
0
votes
1answer
11 views

RNN model weather data for predicting temperature [closed]

I have a 30 years daily weather data. Data Sets parameter are rainfall, cloud amount, humidity, sunshine and temperature. Now, I want to build predictive model. rainfall, cloud amount, humidity and ...
0
votes
0answers
22 views

How can we compute likelihood in recurrent neural networks?

Suppose that we have a recurrent neural network (RNN) with length $T$ for a classification task that generates an output at every time step which is a probability distribution over classes obtained by ...
0
votes
0answers
8 views

How to train a language model with bi lstm layers?

I am trying to understand how to train a LM using bi-LSTM in the case with "stack of bi LSTM". In the case of forward LSTM, we just need to add a classification layer on the top of the last hidden ...
0
votes
0answers
13 views

Keras error in dimensions when predicting

I have trained a Keras LSTM model and was now trying to use it for predictions but for some reason in is giving a dimensions error I cannot find. I processed the data in the same way as the training ...
2
votes
1answer
20 views

What Models should i try for this problem?

I need some advice for a problem i'm working on with automobile data. The vehicles provide a series of codes at every second which are bieng stored, though it can vary how many. For example , at time ...
0
votes
0answers
14 views

How LSTM compare which information is important or not?

I am interested to know, if I have scaled my data between [0,1], and have a vector like [0, 0.001, 0.01, 0.1, 1], is that mean ...
0
votes
1answer
21 views

LSTM get next output with Keras

So I'm learning RNN, and tried to do a prediction LSTM, but I do not understand how the output works. I have this LSTM RNN: ...
1
vote
1answer
36 views

in TensorFlow 2.0, what is the different between LSTM and LSTMCell objects?

I am trying to implement an RNN in TensorFlow 2.0 (beta1). Looking at the layer functions (inherited from Keras) I found: tf.keras.layers.LSTM and ...
0
votes
0answers
15 views

characteristics of time-series datasets

I am working on two different datasets, one of them is a simulated dataset and the other is a real-world dataset. My understanding of RNNs was that they work well with sequential data where a ...
0
votes
0answers
15 views

How to Reduce Overfitting of Deeplearning models on NLP tasks in unbalanced datasets?

I have a binary classification problem, where the number of examples belonging to Class 0 is 20% on average. And the rest 80% of examples fall into Class ...
0
votes
0answers
11 views

Difference between globalmaxpoolin1d() and attention layer

What's the difference between globalmaxpoolin1d() and attention layer?
0
votes
0answers
10 views

How to generate sequence of text(overall trend) by reading stock price

I would like to generate a sequence of text by reading stock price, this sequence text should contain describing the trend of the stock prices and trajectory. There are two types of input sources, ...
0
votes
0answers
38 views

In an RNN, if the gradients don't vanish for long/distant terms, won't the derivative of the error be either divergent to infinity or oscillatory?

P.S. Crosss posted here- https://stats.stackexchange.com/questions/413843/in-an-rnn-if-the-gradients-dont-vanish-for-long-distant-terms-wont-the-deriv, as I've got no answer, I'm asking here: In my ...
0
votes
2answers
51 views

How to predict next visit date based on this data

I have a dataset shown below. Here, status is if visit has been done or not and schedule is if next_action_scheduled. ...
0
votes
1answer
23 views

Understanding Layers in Recurrent Neural Networks for NLP

In convolution neural networks, we have a concept that inner layers learn fine features like lines and edges, while outer layers learn more complex shapes. Do we have any such understanding for ...
0
votes
0answers
23 views

Detect Typical Customer Mistakes in the Shopping Cart

I would like to ask your advice on solving this problem. Problem: There is an online store which sells furniture. There are millions of products on the store like furniture parts and furniture ...
0
votes
1answer
11 views

How to scale exponential data for a regression problem?

I understand that I should be scaling features between (0, 1) before feeding them into a neural network. However, what happens if future data could be larger than my current training data? For ...
9
votes
3answers
174 views

Why do RNNs usually have fewer hidden layers than CNNs?

CNNs can have hundreds of hidden layers and since they are often used with image data, having many layers captures more complexity. However, as far as I have seen, RNNs usually have few layers e.g. ...
0
votes
2answers
35 views

How to tackle a multilabel classification problem

I am trying to build a LSTM model for a multiclass classification problem on textual data. Until now, I have only built a model when one input belongs to one of the categories. What do I do when one ...
0
votes
0answers
20 views

way to add target delay on TimeseriesGenerator from keras.preprocessing.sequence

from deep learning with python book, it created function for data generator. I thought I can do the same using TimeseiresGenerator from keras package but I was not able to add target delay. Is there ...
2
votes
1answer
14 views

What are the equations involved in calculation of the parameters of embedding layer?

I'm trying to perform sentiment analysis on some data using keras.I'm using embedding layer and then LSTM. I know that embedding layer decreases the sparsity of the one hot encodings of the words and ...
0
votes
0answers
39 views

Is it right that ARIMA model is better for short time series and RNN better for long time series?

Is it right that ARIMA model is better for short time series and RNN better for long time series? And why is it so? I watched this question, but did not understand why this is so. Time series ...
1
vote
0answers
33 views

LSTM component dimensions and freedom of design

Like many, I'm learning about LSTMs using this rather clear blog post and to some extent this construction of an RNN cell in Pytorch. I've been studying this diagram: and while I understand the ...
0
votes
1answer
46 views

How do I perform weighted loss in multiple outputs on a same model in Tensorflow?

How do I perform weighted loss in multiple outputs on a same model in Tensorflow? This means I am using a model that is intended to have 3 outputs. I did this ...
0
votes
0answers
12 views

Recurrent Neural Network +LTSM to find the text with High Probability Value

Dataset contains the features such as description, goal , category etc to predict the probability in decimal such as 0.1,0.8 etc, Now in the next step need to find out the text words associated with ...
1
vote
1answer
37 views

correct ML approach

I wanted to get your thoughts on a problem I have been facing. I have daily level product sales information (about 4 years). The sales are affected by the typical factors such as seasonality, day of ...
1
vote
1answer
26 views

MinMaxScaler when LSTM predictions fall outside of training range?

I am using MinMaxScaler on my training set and applying the transformations to my test set and inverse_transform to my model’s outputs. If this were, say, a stock prediction problem, my training set ...
0
votes
1answer
18 views

May I use the same data in several time series intervals?

I am playing with RNNs / LTSMs for a classification task in predicting financial data. I have a time-series going many years back, and are planning to divide it into a number of shorter time-...
1
vote
0answers
53 views

LSTM prediction for many multivariate time series

Let's say we have 8,000 different time series where each of them has 10,000 samples and 25 features. The goal is to have an LSTM sequence to sequence model (using Keras) where one can use a sequence ...
0
votes
0answers
11 views

Why are reservoir computer so useful for hardware implementations

I often read (e.g. here or in this question) that Reservoir Computer (RC) are useful in the field of Neuromorphic Computing where they can serve as efficient implementations of neural networks in ...
1
vote
0answers
31 views

Generate sentences using given data [closed]

I am working on an automated insights generation use case where I want to generate meaningful sentences from given aggregated data. For example, Data: ...
1
vote
2answers
173 views

Training LSTM with different sequence lengths in Keras functional api

I am trying to train an LSTM model using Keras functional API. My training data is of shape: >>> data.shape() (100000,variable_sequence_lengths,295) ...
0
votes
0answers
15 views

What does “large sparse” and “small dense” means?

In a paper I'm reading today it's written : For a fixed parameter count, we discover that large sparse WaveRNNs significantly outperform small dense WaveRNNs and that this relationship holds up ...
0
votes
0answers
20 views

Train LSTM RNN with multiple different sets of time series data in Keras

I am trying to set up a program where an airplane is taking off from one city and flying to another. Depending on a number of factors it can take different routes to get to the city. Since some ...
0
votes
0answers
11 views

What will go wrong if we apply linear or other types of regression to translate sentences between two languages?

Disclaimer: I asked the question at https://stats.stackexchange.com/questions/408463/what-will-go-wrong-if-we-apply-linear-or-other-types-of-regression-to-translate, but didn't get any response, so I'...
1
vote
1answer
26 views

How can I go about building a model for large number of outputs?

I have previously worked on small-scale feedforward neural network problems. But I have started working on a new project where the goal is to predict air quality in 25 locations throughout the ...
1
vote
0answers
58 views

High Training Accuracy, Poor Validation, Test Accuracy

I am a beginner exploring Deep learning. I am trying to train a classifier (9 classes) with images as the input to my CNN followed by Bidirectional LSTM architecture. My model rapidly achieves a ...
1
vote
1answer
35 views

Is it possible that a CNN has better accuracy than RNN in word classification?

So I found something strange once I compared the accuracy of the prediction of a class for a question between a CNN and an RNN (GRU). The CNN achieved 0.87 accuracy over the RNN (GRU) with 0.7520 ...
2
votes
1answer
66 views

Time Series Forecasting for Multiple Customers using one RNN

I have a product which has univariate and also multivariate time series data from multiple customers. I have variable amount of data available. Ranging between couple of years to couple of months. ...
3
votes
0answers
26 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
1
vote
0answers
44 views

Using TF Dataset API to process sequences for stateful RNN

I am trying to use the TensorFlow (v1.13) Dataset API to save and load long sequences for a stateful RNN. Basically, lets say I have n_seq sequences, each fixed ...
0
votes
0answers
28 views

Time series output of LSTM network has a much lower scale than the input scale

I'm trying to use an LSTM network to predict a sequence of a time series variable. I'm trying to predict a sequence of 3 elements based on the sequence of the previous 6 elements. The Keras code that ...
0
votes
0answers
22 views

Pytorch's pack_padded_sequence in Tensorflow?

If we do not use pack_padded_sequence of Pytorch, what will happen to the eval result? How to implement Pytorch's ...
1
vote
0answers
22 views

How Tensorflow text prediction predicts without softmax activation

In the Colab notebook here: RNN text generation in def generate_text(), there is ...
1
vote
0answers
74 views

How to apply an RNN to forecast non-stationary time series?

Is it possible to predict a time series which is non-stationary, in the sense that, the dependent variable Y have an increasing trend? Therefore, the highest value of $Y$ in the training set may be ...
0
votes
0answers
23 views

Hybrid Model : RNN + MLP + RNN

I am trying to develop a model, as follow: an RNN with three LSTM takes in the input (5|1|54), the 5 previous days and 54 feature. In the end of the first RNN I would like to take the mean and std of ...
1
vote
1answer
47 views

How to implement this CNN architecture in Keras

I am trying to implement in Keras the CNN architecture used by Rajpurkar et al and illustrated below: I am particularly confused about that max pool that is shown ...
2
votes
1answer
93 views

How to design batches in a stateful RNN

I am using TF Eager to train a stateful RNN (GRU). I have several variable length time sequences about 1 minute long which I split into windows of length 1s. In TF Eager, like in Keras, if ...
0
votes
1answer
19 views

Why don't we gradually update the activation parameters in RNN from one activation to the next as the network is learning more?

I'm very new to (unidirectional, vanilla) RNN and sequence modeling in general, and all I understood about the motivation on having the connection between two successive hidden layers/activation is ...
0
votes
0answers
30 views

Duplicate QUORA question detection:Kaggle Dataset

I have tried to use 2 BILSTMs along with the attention layer but the validation accuracy is not improving at all. Could anyone suggest an alternative to increase the accuracy? Layer structuring: <...