Questions tagged [glorot-initialization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
5 votes
1 answer

How/What to initialize the hidden states in RNN sequence-to-sequence models?

In an RNN sequence-to-sequence model, the encode input hidden states and the output's hidden states needs to be initialized before training. What values should we initialize them with? How should we ...
alvas's user avatar
  • 2,340
3 votes
1 answer

What is glorot uniform?

While reading Bengio and Glorot, came across this, but after going through paper, intution about the algorithm was not so deep, care to explain why was this needed and where they needed to implement ...
TheExorcist's user avatar
2 votes
0 answers

TF: What is the difference between the 'kernel weights' and the 'recurrent kernel weights' in LSTMs/GRUs?

Context: I am trying to understand the differences between the GRU/LSTM cells from tensorflow and pytorch (for research reproducibility) and noticed that TensorFlow differentiates between the ...
Robin van Hoorn's user avatar
1 vote
0 answers

confusion with Xavier Initiliazation definition

When researching online, I keep finding that Xavier/Glorot initialization is: however, the original paper by Glorot said that this was a common initialization strategy that they soon found did not ...
tom394's user avatar
  • 11
0 votes
1 answer

How is the property in eq 15 obtained for Xavier initialization

I am new in this field so please be gentle with terminology. In the original paper; "Understanding the difficulty of training deep feedforward neural networks", I dont understand how equation 15 is ...
Laughing Man's user avatar