# What does the one function $\mathbf{1}_{i,y^{(t)}}$ exactly mean in backward propagation of RNN in the book "Deep learning" of Bengio

It confused me for a long time what is $$\mathbf{1}_{i,y^{(t)}}$$ exactly mean in (10.18) below.

It is in the Chapter 10 on RNN of the book LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep learning." nature 521.7553 (2015): 436.

Because the $${i,y^{(t)}}$$ doesn't seem to be a condition while the author has introduced a condition function as $$\mathbf{1}_\text{condition} = 1$$ if condition is true or $$=0$$ if the condition is false.

It is computing the gradient of the log-likelihood with backward propagation in recurrent neural network.

Instead of $$\hat{y}_i^{(t)}− \mathbf{1}_{i,y^{(t)}}$$, I found my solution might be $$(\hat{y}^{(t)}_i-y_i^{(t)})\dfrac{\exp(o_i^{(t)})\sum_j\exp(z_j)-(\exp(z_i))^2}{(\sum_j\exp(z_j))^2}$$ It is based on the prior assumption of Gaussian distribution with $$\sigma=1$$

The gradient $$\nabla_{o^{(t)}}L$$ on the outputs at time step $$t$$, for all $$i$$, $$t$$, is as follows: $$(\nabla_{o(t)}L)_i = \dfrac{\partial L}{\partial o_i^{(t)}} = \dfrac{\partial L}{\partial L^{(t)}}\dfrac{\partial L^{(t)}}{\partial o_i^{(t)}} = \hat{y}_i^{(t)}− \mathbf{1}_{i,y^{(t)}}. \qquad\qquad (10.18)$$

While, the corresponding notes are:

\begin{align*} \boldsymbol{a}^{(t)} &= \boldsymbol{b} +\boldsymbol{Wh}^{(t−1)} + \boldsymbol{Ux}^{(t)} &(10.8)\\ \boldsymbol{h}^{(t)} &= \tanh(\boldsymbol{a}^{(t)}) &(10.9)\\ \boldsymbol{o}^{(t)} &= \boldsymbol{c} + \boldsymbol{V h}^{(t)} &(10.10)\\ \hat{\boldsymbol{y}}^{(t)} &= \text{softmax}(\boldsymbol{o}^{(t)}) &(10.11) \end{align*}

And I wonder which version of softmax function is used here? $$\text{softmax}(\boldsymbol{z})=\dfrac{\exp(z_i)}{\sum_j\exp(z_j)}$$ or $$\text{softmax}(\boldsymbol{z})=\dfrac{\exp(z_i-\max_kz_k)}{\sum_j\exp(z_j-\max_kz_k)}$$ ?

Thank you very much!

The authors or the publisher may have corrected the inconsistent notation. The current version online looks like below:

Here the $\mathbf{1}_{\text{condition}}$ is the indicator function, as you mentioned.

Regarding the two version of softmax, use the first one for calculating gradient. The second version is for improving numerical stability, and mathmatically it should yield the exact same value as the first version.

• Thank you for your help! I have a further question that shouldn't $y^{(t)}$ be a vector $\boldsymbol{y}^{t}$ with the same length as $\hat{\boldsymbol{y}}^{(t)}$? Then how could it be equal to a scalar $i$?
– gph
Commented Sep 20, 2018 at 2:59
• And the expression of $L$? I have thought it was based on assumption that :$p(y|x;\theta)\sim \mathcal{N}(y,\mu=\hat{y},\sigma=1)$, i.e Gaussian distribution. So $L=\sum_t\frac{1}{2}\Vert\hat{\boldsymbol{y}}^{(t)}-\boldsymbol{y}^{(t)}\Vert^2+Constant$ But it seems that here the authors use the cross entropy loss like $L=\sum_t \boldsymbol{y}^{(t)}t\log{\hat{\boldsymbol{y}}^{(t)}}$? So do you know what is the expression of loss $L$ here?Thank you!
– gph
Commented Sep 20, 2018 at 3:16
• $L$ is the cross-entropy loss defined in (10.14). For the indicator function, you are right that the condition is "scalar $i$"=="vector $y^{t}$", in this case the indicator function returns a vector whose k'th elements are indicator of whether that element in $i$ equals the k'th element of $y^{t}$. Similar like you compare a 1-D numpy array with a scalar in python. Commented Sep 20, 2018 at 3:37
• Thank you for help! I was too impatient to ask the question when I hadn't read the chapters before RNN because I had thought it simple. I read the underlying chapter again and wrote an answer below to prove the equation (10.18). By the way, I think $y^{(t)}$ here is a scalar at certain time t.
– gph
Commented Sep 21, 2018 at 7:34
• you are right. I think the author means $t^{(t)}$ to be a scalar. I was thinking it as a one hot encoding, but it's not here. Commented Sep 21, 2018 at 15:20

I am too impatient to ask the question when I haven't read the chapters before RNN because I have thought it simple. Soon I found it more complicated than I had thought.

Then I read the underlying chapter again and I think I can answer my own question now.

The expression of (10.18) is as the answer by @user12075:

the authors have corrected the notion $$\mathbf{1}_{i,y^{(t)}}$$ to $$\mathbf{1}_{i=y^{(t)}}$$.

Then I want to prove the new version of (10.18).

If there is something I have misunderstood, please tell me! Thank you!

First, I'll explain the loss function and underlying concepts：

According to (10.14): $$$$L=\sum_tL^{(t)}=-\sum_t\log p_{model}(y^{(t)}|\{\boldsymbol{x}^{(1)},\cdots,\boldsymbol{x}^{(t)}\}) \label{loss}$$$$

at each time step t, label $$y^{(t)}$$ is a scalar, for example, it is a multiclass classification with n classes, then $$y^{(t)}\in \{1,2,\cdots,n\}$$

However at each time step t, output of the model $$\hat{\boldsymbol {y}}^{(t)}$$ is a vector with length n, instead of a scalar approximate to the scalar $$y^{(t)}$$,it is represented as the probabilities of $$y^{(t)}$$ taking different values.

That is

\begin{align} \hat{\boldsymbol {y}}^{(t)}=& \begin{bmatrix} p_{model}(y^{(t)}=1|\{\boldsymbol{x}^{(1)},\cdots,\boldsymbol{x}^{(t)}\})\\ p_{model}(y^{(t)}=2|\{\boldsymbol{x}^{(1)},\cdots,\boldsymbol{x}^{(t)}\})\\ \vdots\\ p_{model}(y^{(t)}=n|\{\boldsymbol{x}^{(1)},\cdots,\boldsymbol{x}^{(t)}\}) \end{bmatrix}\\ \text{for short is:}& \begin{bmatrix} p(y^{(t)}=1|X;\Theta)\\ p(y^{(t)}=2|X;\Theta)\\ \vdots\\ p(y^{(t)}=n|X;\Theta) \end{bmatrix} \end{align}

And in the training, value of $$y^{(t)}$$ at each t is already known.

So in (10.14), we assume $$t=3,n=10$$, $$y^{(1)}=3,y^{(2)}=5,y^{(3)}=4$$

then \begin{align} L&=-\log p(y^{(1)}=3|X)-\log p(y^{(2)}=5|X)-\log p(y^{(3)}=4|X)\\ &=-\log \hat{\boldsymbol{y}}^{(1)}_3 -\log \hat{\boldsymbol{y}}^{(2)}_5 -\log \hat{\boldsymbol{y}}^{(3)}_4 \end{align}

While the subscript of $$\hat{\boldsymbol{y}}^{(t)}$$ denotes the index of element of the vector

So the general expression is:

\begin{align} L&=-\sum_t \log p_{model}(y^{(t)}|X)\\ &=-\sum_t \log\hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}\label{grad_L_o} \qquad\qquad (1) \end{align}

Now we can begin to verify the equation (10.18) on the book:

\begin{align} (\nabla_{o(t)}L)_i &= \frac{\partial L}{\partial o_i^{(t)}}\\ &= \frac{\partial L}{\partial L^{(t)}}\frac{\partial L^{(t)}}{\partial o_i^{(t)}}\\ &= \hat{y}_i^{(t)} - \mathbf{1}_{i=y^{(t)}}. \qquad\qquad (10.18) \end{align}

Note: $$i=1,2,\cdots,n,\quad y^{(t)}\in \{1,2,\cdots,n\}$$

And \begin{align} \boldsymbol{a}^{(t)} &= \boldsymbol{b} +\boldsymbol{Wh}^{(t−1)} + \boldsymbol{Ux}^{(t)} &(10.8)\\ \boldsymbol{h}^{(t)} &= \tanh(\boldsymbol{a}^{(t)}) &(10.9)\\ \boldsymbol{o}^{(t)} &= \boldsymbol{c} + \boldsymbol{V h}^{(t)} &(10.10)\\ \hat{\boldsymbol{y}}^{(t)} &= \text{softmax}(\boldsymbol{o}^{(t)}) &(10.11) \end{align} While $$$$(\text{softmax}(\boldsymbol{x}))_i := \frac{\exp(x_i)}{\sum_j \exp(x_j)}$$$$ So $$$$\hat{\boldsymbol{y}}_i^{(t)} = \frac{\exp(o_i^{(t)})}{\sum_j \exp(o_j^{(t)})}$$$$

by(1), we begin to compute the following equation: \begin{align} (\nabla_{o(t)}L)_i =- \frac{\partial \log\hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}}\label{grad_L_o_i} \qquad\qquad (2) \end{align}

We compute each item in the sum respectively: \begin{align} \frac{\partial \log\hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}} =\frac{1}{\hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}\cdot \frac{\partial \hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}} \end{align}

Here are two cases to be considered for $$\frac{\partial \hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}}$$:

$$i=y^{(t)}$$ vs $$i\ne y^{(t)}$$

When $$i=y^{(t)}$$:

\begin{align} \frac{\partial \hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}} &=\frac{\partial \hat{\boldsymbol{y}}^{(t)}_i}{\partial o_i^{(t)}}\\ &=\frac{\exp(o_i^{(t)})\cdot\sum_j \exp(o_j^{(t)})-\exp(o_i^{(t)})\cdot\exp(o_i^{(t)})}{(\sum_j \exp(o_j^{(t)}))^2}\\ &=\frac{\exp(o_i^{(t)})}{\sum_j \exp(o_j^{(t)})} - (\frac{\exp(o_i^{(t)})}{\sum_j \exp(o_j^{(t)})})^2\\ &=\hat{\boldsymbol{y}}^{(t)}_i - (\hat{\boldsymbol{y}}^{(t)}_i)^2 \end{align} So \begin{align} \frac{\partial \log\hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}} &=\frac{1}{\hat{\boldsymbol{y}}^{(t)}_i}\cdot \frac{\partial \hat{\boldsymbol{y}}^{(t)}_i}{\partial o_i^{(t)}}\\ & = \frac{1}{\hat{\boldsymbol{y}}^{(t)}_i}\cdot (\hat{\boldsymbol{y}}^{(t)}_i - (\hat{\boldsymbol{y}}^{(t)}_i)^2)\\ &=1 - \hat{\boldsymbol{y}}^{(t)}_i \end{align}

When $$i\ne y^{(t)}$$:

\begin{align} \frac{\partial \hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}} &=\frac{-\exp(o_i^{(t)})\cdot\exp(o_{y^{(t)}}^{(t)})} {(\sum_j \exp(o_j^{(t)}))^2}\\ &=- \frac{\exp(o_i^{(t)})}{\sum_j \exp(o_j^{(t)})}\cdot \frac{\exp(o_{y^{(t)}}^{(t)})}{\sum_j \exp(o_j^{(t)})}\\ &= - \hat{\boldsymbol{y}}^{(t)}_i\cdot \hat{\boldsymbol{y}}^{(t)}_{y^{(t)}} \end{align} So \begin{align} \frac{\partial \log\hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}} &=\frac{1}{\hat{\boldsymbol{y}}^{(t)}_i}\cdot \frac{\partial \hat{\boldsymbol{y}}^{(t)}_i}{\partial o_i^{(t)}}\\ & = \frac{1}{\hat{\boldsymbol{y}}^{(t)}_i}\cdot (- \hat{\boldsymbol{y}}^{(t)}_i\cdot \hat{\boldsymbol{y}}^{(t)}_{y^{(t)}})\\ &=- \hat{\boldsymbol{y}}^{(t)}_i \end{align}

So substitute into (2), we got : \begin{align} (\nabla_{o(t)}L)_i &= \begin{cases} \hat{\boldsymbol{y}}^{(t)}_i-1 &\text{If } i=y^{(t)},\\ \hat{\boldsymbol{y}}^{(t)}_i &\text{If } i\ne y^{(t)} \end{cases}\\ &=\hat{y}_i^{(t)} - \mathbf{1}_{i=y^{(t)}}. \qquad\qquad \text{which is (10.18)} \end{align}

So (10.18) proved.