1
$\begingroup$

When researching online, I keep finding that Xavier/Glorot initialization is:

enter image description here enter image description here

however, the original paper by Glorot said that this was a common initialization strategy that they soon found did not work great:

enter image description here

and instead they came up with a new initialization scheme:

enter image description here

To me, these look like two completely different schemes. The first only is only a function of the number of inputs to a layer, and the second is a function of both the number of inputs AND the number of outputs. I don't know if the normal vs uniform distribution makes up for the difference.

So my questions are:

  • which one is the ACTUAL Xavier initialization?
  • what are the differences between the two schemes?
  • which one do I use?
  • what did not work about the first scheme that prompted Glorot to come up with the second one?
$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.