While reading Bengio and Glorot, came across this, but after going through paper, intution about the algorithm was not so deep, care to explain why was this needed and where they needed to implement this ?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Here is a brief discussion of the Xavier initialization.
The goal of Xavier Initialization is to initialize the weights such that the variance of the activations are the same across every layer. This constant variance helps prevent the gradient from exploding or vanishing.
Also check this for a slightly longer discussion on the topic by the same instructor.
