0
$\begingroup$

For my CNN model I tried some of the weight initializers such as "truncated normal initializer","random normal initializer","glorot normal initializer","glorot uniform initializer", my questions are:

1- I need the equations for each of them, for the two Xavier initializers I found more than one type of equation for that I confused which one is true but for other two I didn't found any equation?

2- Is the range of values are between 0 and 1 for all of them? are they considered a gaussian?

$\endgroup$

1 Answer 1

1
+50
$\begingroup$

enter image description here

2 - Not necessarily between 0 an 1. Example: If you see the normal glorot formulae, The sigma_i value will entirely depend on fan_in and fan_out. So, it need not be centred at 0.

EDIT 1:

enter image description here

EDIT 2:

enter image description here

enter image description here

Research paper - weight initialization by Xavier

Credits: Also, check out this and this Hope it helps.

$\endgroup$
7
  • $\begingroup$ really thank you for your help, you deserve the bounty, but please could edit the answer and add detail about the equations? and could you tell me the reference of these equations because i need them for academic purpose? $\endgroup$
    – Hunar
    Commented Mar 8, 2019 at 11:23
  • $\begingroup$ Sure, I'm working on it $\endgroup$ Commented Mar 8, 2019 at 13:19
  • $\begingroup$ I've made the edit!! See if its any helful $\endgroup$ Commented Mar 8, 2019 at 18:05
  • $\begingroup$ Thank you dear, it's very helpful, but I need the explanation of each symbol used in the formulas, can you do edit and add them? for example what is fan_in and fan_out. $\endgroup$
    – Hunar
    Commented Mar 9, 2019 at 6:37
  • 1
    $\begingroup$ Yes, Now I have added it in edit2 $\endgroup$ Commented Mar 9, 2019 at 9:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.