I am trying to understand gradient calculation for skip gram with softmax output and cross entropy loss.
I am referring these articles: 1, 2, 3.
The all calculate the error as follows:
$$E=-\sum_{c=1}^Cu_{j_{c^*}}+C.log\left(\sum_{j'=1}^v e^uj'\right)$$
Step 1: Then they calculate the gradient of Error $E$ with respect to $u_j$ as follows:
$$\frac{\partial E}{\partial u_j}=-\sum_{c=1}^Cu_{j_{c^*}}+C.\frac{1}{\sum_{j'=1}^v e^{u_j^*}}\frac{\partial }{\partial u_j}\left(\sum_{j'=1}^v e^uj'\right)$$
Q1. Why the first term above $-\sum_{c=1}^Cu_{j_{c^*}}$ left untouched? That is, why they have not taken its gradient with respect to $u_j$?
Step2: In next step, they do:
$$\frac{\partial E}{\partial u_j}=-\sum_{c=1}^C 1+\sum_{j=1}^V y_j$$
Q2. I am unable to get how both terms in above sum are obtained.
Step 3: Finally the articles do:
$$\frac{\partial E}{\partial u_j}=y_j-t_j$$
Q3. Again I did not get how this step is obtained.
Can someone please tell me what I am missing here?
