# Tag Info

### When to use GRU over LSTM?

GRUs and LSTMs utilize different approaches toward gating information to prevent the vanishing gradient problem. Here are the main points comparing the two: The GRU unit controls the flow of ...
• 1,989

### When to use GRU over LSTM?

*To complement already great answers above. From my experience, GRUs train faster and perform better than LSTMs on less training data if you are doing language modeling (not sure about other tasks). ...
• 2,157
Accepted

### Time Series prediction using LSTMs: Importance of making time series stationary

In general time series are not really different from other machine learning problems - you want your test set to 'look like' your training set, because you want the model you learned on your training ...
• 2,248
Accepted

### How to feed LSTM with different input array sizes?

The easiest way is to use Padding and Masking. There are three general ways to handle variable-length sequences: Padding and masking (which can be used for (3)), Batch size = 1, and Batch size > 1,...
• 9,342

### Validation loss is not decreasing

The model is overfitting right from epoch 10, the validation loss is increasing while the training loss is decreasing. Dealing with such a Model: Data Preprocessing: Standardizing and Normalizing the ...
• 675

### When to use GRU over LSTM?

FULL GRU Unit $\tilde{c}_t = \tanh(W_c [G_r * c_{t-1}, x_t ] + b_c)$ $G_u = \sigma(W_u [ c_{t-1}, x_t ] + b_u)$ $G_r = \sigma(W_r [ c_{t-1}, x_t ] + b_r)$ \$ c_t = G_u * \tilde{c}_t + (1 - ...
• 351

### What's the difference between the cell and hidden state in LSTM?

In short: Cell state: Long term memory of the model, only part of LSTM models Hidden state: Working memory, part of LSTM and RNN models Additional Information RNN and vanishing/exploding gradients ...
• 2,058

### Dropout on which layers of LSTM?

I prefer not to add drop out in LSTM cells for one specific and clear reason. LSTMs are good for long terms but an important ...
• 14.1k

### When to use GRU over LSTM?

This answer actually lies on the dataset and the use case. It's hard to tell definitively which is better. GRU exposes the complete memory unlike LSTM, so applications which that acts as advantage ...
• 2,346
Accepted

### Multi-dimentional and multivariate Time-Series forecast (RNN/LSTM) Keras

The input shape for an LSTM must be (num_samples, num_time_steps, num_features). In your example case, combining both cities as input, ...
• 14.9k
Accepted

### What is LSTM, BiLSTM and when to use them?

RNN architectures like LSTM and BiLSTM are used in occasions where the learning problem is ...
• 14.1k
Accepted

### Sliding window leads to overfitting in LSTM?

Although the previous answer by @Imran is correct, I feel it necessary to add a caveat: there are applications out there where people do feed a sliding window in to an LSTM. For example, here, for ...
• 2,011
Accepted

### Activation function between LSTM layers

Simple explanation with images We know that an activation is required between matrix multiplications to afford a neural network the ability to model non-linear processes. A classical LSTM cell ...
• 14.9k

### So what's the catch with LSTM?

You are right that LSTMs work very well for some problems, but some of the drawbacks are: LSTMs take longer to train LSTMs require more memory to train LSTMs are easy to overfit Dropout is much ...
• 2,381

### What is the best method for classification of time series data? Should I use LSTM or a different method?

I would not use the word "best" but LSTM-RNN are very powerful when dealing with timeseries, simply because they can store information about previous values and exploit the time dependencies ...
• 3,950
Accepted

### Prediction interval around LSTM time series forecast

Directly, this is not possible. However, if you model it in a different way you can get out confidence intervals. You could instead of a normal regression approach it as estimating a continuous ...
• 9,378
Accepted

### Keras LSTM with 1D time series

LSTM layers require data of a different shape. From your description, I understand the starting dataset to have 3125 rows and 1000 columns, where each row is one time-step. The target variable should ...
• 14.9k

### When to use Stateful LSTM?

As for stateful LSTM and its understanding, refer to here. Quoting an answer from there: "I’m given a big sequence (e.g. Time Series) and I split it into smaller sequences to construct my input ...
• 3,950

### What does the output of model.predict function from Keras mean?

The output of a neural network will never, by default, be binary - i.e. zeros or ones. The network works with continuous values (not discrete) in order to optimise the loss more freely in the ...
• 14.9k
Accepted

### What is the job of "RepeatVector" and "TimeDistributed"?

tf.keras.layers.RepeatVector According to the docs : Repeats the input n times. They have also provided an example : ...
• 2,200
Accepted

### Checking for stationarity in LSTM

In principle no you do not need to check for stationarity nor correct for it when you are using an LSTM. The thing about stationarity is that it makes prediction tasks much more efficient, and stable....
• 722

### Binary classification of every time series step based on past and future values

You are facing a very common problem: handling imbalanced data. For neural networks, typical procedures are: Having the proper metrics: global accuracy should not be used. Oversampling the minority ...
• 1,668
Accepted

### Batch Size of Stateful LSTM in keras

I solved the problem this way: I realized that I needed to find the HCF (highest common factor) of both the length of x_train ...
• 420
Accepted

### What's the difference of stateless LSTM and a normal feed-forward NN?

Theoretically a stateless LSTM gives the same result as a statefull LSTM, but there are few pros and cons between them. A stateless LSTM requires you to structure your data in a particular way, in ...

### What is the job of "RepeatVector" and "TimeDistributed"?

For encoder-decoder, your input is squashed into a single feature vector, if you want your output to regenerate the same dimension as the original input, you are "artificially" converting this feature ...
• 201
Accepted

### Value Error: Operands could not be broadcast together with shapes - LSTM

The answer that OP provided is correct, yet I would like to elaborate a little more on it, in an attempt to shed more light. First of all, you have to understand what is performed by the call ...
• 3,950

From What are the advantages of stacking multiple LSTMs? (I'll only update the answer there): From {1}: While it is not theoretically clear what is the additional power gained by the ...
• 5,740

### Validation loss is not decreasing

Yes this is an overfitting problem since your curve shows point of inflection. This is a sign of very large number of epochs. In this case, model could be stopped at point of inflection or the number ...
Accepted

### How to use Embedding() with 3D tensor in Keras?

I'm not entirely sure if this is the cleanest solution but I stitched everything together. Each of the 10 word positions get their own input but that shouldn't be too much of a problem. The idea is to ...
• 9,378