Timeline for Layer normalization details in GPT-2
Current License: CC BY-SA 4.0
2 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 30, 2021 at 16:47 | comment | added | rampatowl | Hi Ashwin, thanks. I understand where LayerNorm occurs in the architecture. I am asking about the computational details: which set of scalars exactly are used to compute each sample mean and standard dev, and how many such normalization statistics are computed in my example. This is not clear to me after reading the papers, and if it is obvious to you please be explicit about the details. | |
| Jan 30, 2021 at 13:21 | history | answered | Ashwin Geet D'Sa | CC BY-SA 4.0 |