4
$\begingroup$

While reading Bengio and Glorot, came across this, but after going through paper, intution about the algorithm was not so deep, care to explain why was this needed and where they needed to implement this ?

$\endgroup$

1 Answer 1

4
$\begingroup$

Here is a brief discussion of the Xavier initialization.

The goal of Xavier Initialization is to initialize the weights such that the variance of the activations are the same across every layer. This constant variance helps prevent the gradient from exploding or vanishing.

Also check this for a slightly longer discussion on the topic by the same instructor.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.