Questions tagged [glorot-initialization]
The glorot-initialization tag has no usage guidance.
5
questions
1
vote
0
answers
12
views
confusion with Xavier Initiliazation definition
When researching online, I keep finding that Xavier/Glorot initialization is:
however, the original paper by Glorot said that this was a common initialization strategy that they soon found did not ...
- 11
2
votes
0
answers
74
views
TF: What is the difference between the 'kernel weights' and the 'recurrent kernel weights' in LSTMs/GRUs?
Context:
I am trying to understand the differences between the GRU/LSTM cells from tensorflow and pytorch (for research reproducibility) and noticed that TensorFlow differentiates between the ...
- 164
3
votes
1
answer
4k
views
What is glorot uniform?
While reading Bengio and Glorot, came across this, but after going through paper, intution about the algorithm was not so deep, care to explain why was this needed and where they needed to implement ...
- 131
0
votes
1
answer
30
views
How is the property in eq 15 obtained for Xavier initialization
I am new in this field so please be gentle with terminology.
In the original paper; "Understanding the difficulty of training deep feedforward neural networks", I dont understand how equation 15 is ...
5
votes
1
answer
9k
views
How/What to initialize the hidden states in RNN sequence-to-sequence models?
In an RNN sequence-to-sequence model, the encode input hidden states and the output's hidden states needs to be initialized before training.
What values should we initialize them with? How should we ...
- 2,340