Questions tagged [glorot-initialization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
39 views

What is glorot uniform?

While reading Bengio and Glorot, came across this, but after going through paper, intution about the algorithm was not so deep, care to explain why was this needed and where they needed to implement ...
0
votes
1answer
24 views

How is the property in eq 15 obtained for Xavier initialization

I am new in this field so please be gentle with terminology. In the original paper; "Understanding the difficulty of training deep feedforward neural networks", I dont understand how equation 15 is ...
4
votes
1answer
7k views

How/What to initialize the hidden states in RNN sequence-to-sequence models?

In an RNN sequence-to-sequence model, the encode input hidden states and the output's hidden states needs to be initialized before training. What values should we initialize them with? How should we ...