What does the one function $\mathbf{1}_{i,y^{(t)}}$ exactly mean in backward propagation of RNN in the book "Deep learning" of Bengio

Question

It confused me for a long time what is $\mathbf{1}_{i,y^{(t)}}$ exactly mean in (10.18) below.

It is in the Chapter 10 on RNN of the book LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep learning." nature 521.7553 (2015): 436.

Because the ${i,y^{(t)}}$ doesn't seem to be a condition while the author has introduced a condition function as $\mathbf{1}_\text{condition} = 1$ if condition is true or $=0$ if the condition is false.

It is computing the gradient of the log-likelihood with backward propagation in recurrent neural network.

Instead of $\hat{y}_i^{(t)}− \mathbf{1}_{i,y^{(t)}}$, I found my solution might be $$ (\hat{y}^{(t)}_i-y_i^{(t)})\dfrac{\exp(o_i^{(t)})\sum_j\exp(z_j)-(\exp(z_i))^2}{(\sum_j\exp(z_j))^2} $$ It is based on the prior assumption of Gaussian distribution with $\sigma=1$

Below is the equation I am asking about:

The gradient $\nabla_{o^{(t)}}L$ on the outputs at time step $t$, for all $i$, $t$, is as follows: $$ (\nabla_{o(t)}L)_i = \dfrac{\partial L}{\partial o_i^{(t)}} = \dfrac{\partial L}{\partial L^{(t)}}\dfrac{\partial L^{(t)}}{\partial o_i^{(t)}} = \hat{y}_i^{(t)}− \mathbf{1}_{i,y^{(t)}}. \qquad\qquad (10.18) $$

While, the corresponding notes are:

$$ \begin{align*} \boldsymbol{a}^{(t)} &= \boldsymbol{b} +\boldsymbol{Wh}^{(t−1)} + \boldsymbol{Ux}^{(t)} &(10.8)\\ \boldsymbol{h}^{(t)} &= \tanh(\boldsymbol{a}^{(t)}) &(10.9)\\ \boldsymbol{o}^{(t)} &= \boldsymbol{c} + \boldsymbol{V h}^{(t)} &(10.10)\\ \hat{\boldsymbol{y}}^{(t)} &= \text{softmax}(\boldsymbol{o}^{(t)}) &(10.11) \end{align*} $$

And I wonder which version of softmax function is used here? $$ \text{softmax}(\boldsymbol{z})=\dfrac{\exp(z_i)}{\sum_j\exp(z_j)} $$ or $$ \text{softmax}(\boldsymbol{z})=\dfrac{\exp(z_i-\max_kz_k)}{\sum_j\exp(z_j-\max_kz_k)} $$ ?

Thank you very much!

Answer 1

The authors or the publisher may have corrected the inconsistent notation. The current version online looks like below:

Here the $\mathbf{1}_{\text{condition}}$ is the indicator function, as you mentioned.

Regarding the two version of softmax, use the first one for calculating gradient. The second version is for improving numerical stability, and mathmatically it should yield the exact same value as the first version.

Answer 2

I am too impatient to ask the question when I haven't read the chapters before RNN because I have thought it simple. Soon I found it more complicated than I had thought.

Then I read the underlying chapter again and I think I can answer my own question now.

The expression of (10.18) is as the answer by @user12075:

the authors have corrected the notion $\mathbf{1}_{i,y^{(t)}}$ to $\mathbf{1}_{i=y^{(t)}}$.

Then I want to prove the new version of (10.18).

If there is something I have misunderstood, please tell me! Thank you!

First, I'll explain the loss function and underlying concepts：

According to (10.14): \begin{equation} L=\sum_tL^{(t)}=-\sum_t\log p_{model}(y^{(t)}|\{\boldsymbol{x}^{(1)},\cdots,\boldsymbol{x}^{(t)}\}) \label{loss} \end{equation}

at each time step t, label $y^{(t)}$ is a scalar, for example, it is a multiclass classification with n classes, then $y^{(t)}\in \{1,2,\cdots,n\}$

However at each time step t, output of the model $\hat{\boldsymbol {y}}^{(t)}$ is a vector with length n, instead of a scalar approximate to the scalar $y^{(t)}$,it is represented as the probabilities of $y^{(t)}$ taking different values.

That is

\begin{align} \hat{\boldsymbol {y}}^{(t)}=& \begin{bmatrix} p_{model}(y^{(t)}=1|\{\boldsymbol{x}^{(1)},\cdots,\boldsymbol{x}^{(t)}\})\\ p_{model}(y^{(t)}=2|\{\boldsymbol{x}^{(1)},\cdots,\boldsymbol{x}^{(t)}\})\\ \vdots\\ p_{model}(y^{(t)}=n|\{\boldsymbol{x}^{(1)},\cdots,\boldsymbol{x}^{(t)}\}) \end{bmatrix}\\ \text{for short is:}& \begin{bmatrix} p(y^{(t)}=1|X;\Theta)\\ p(y^{(t)}=2|X;\Theta)\\ \vdots\\ p(y^{(t)}=n|X;\Theta) \end{bmatrix} \end{align}

And in the training, value of $y^{(t)}$ at each t is already known.

So in (10.14), we assume $ t=3,n=10 $, $ y^{(1)}=3,y^{(2)}=5,y^{(3)}=4 $

then \begin{align} L&=-\log p(y^{(1)}=3|X)-\log p(y^{(2)}=5|X)-\log p(y^{(3)}=4|X)\\ &=-\log \hat{\boldsymbol{y}}^{(1)}_3 -\log \hat{\boldsymbol{y}}^{(2)}_5 -\log \hat{\boldsymbol{y}}^{(3)}_4 \end{align}

While the subscript of $\hat{\boldsymbol{y}}^{(t)}$ denotes the index of element of the vector

So the general expression is:

\begin{align} L&=-\sum_t \log p_{model}(y^{(t)}|X)\\ &=-\sum_t \log\hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}\label{grad_L_o} \qquad\qquad (1) \end{align}

Now we can begin to verify the equation (10.18) on the book:

\begin{align} (\nabla_{o(t)}L)_i &= \frac{\partial L}{\partial o_i^{(t)}}\\ &= \frac{\partial L}{\partial L^{(t)}}\frac{\partial L^{(t)}}{\partial o_i^{(t)}}\\ &= \hat{y}_i^{(t)} - \mathbf{1}_{i=y^{(t)}}. \qquad\qquad (10.18) \end{align}

Note: $i=1,2,\cdots,n,\quad y^{(t)}\in \{1,2,\cdots,n\}$

And \begin{align} \boldsymbol{a}^{(t)} &= \boldsymbol{b} +\boldsymbol{Wh}^{(t−1)} + \boldsymbol{Ux}^{(t)} &(10.8)\\ \boldsymbol{h}^{(t)} &= \tanh(\boldsymbol{a}^{(t)}) &(10.9)\\ \boldsymbol{o}^{(t)} &= \boldsymbol{c} + \boldsymbol{V h}^{(t)} &(10.10)\\ \hat{\boldsymbol{y}}^{(t)} &= \text{softmax}(\boldsymbol{o}^{(t)}) &(10.11) \end{align} While \begin{equation} (\text{softmax}(\boldsymbol{x}))_i := \frac{\exp(x_i)}{\sum_j \exp(x_j)} \end{equation} So \begin{equation} \hat{\boldsymbol{y}}_i^{(t)} = \frac{\exp(o_i^{(t)})}{\sum_j \exp(o_j^{(t)})} \end{equation}

by(1), we begin to compute the following equation: \begin{align} (\nabla_{o(t)}L)_i =- \frac{\partial \log\hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}}\label{grad_L_o_i} \qquad\qquad (2) \end{align}

We compute each item in the sum respectively: \begin{align} \frac{\partial \log\hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}} =\frac{1}{\hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}\cdot \frac{\partial \hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}} \end{align}

Here are two cases to be considered for $\frac{\partial \hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}}$:

$i=y^{(t)}$ vs $i\ne y^{(t)}$

When $i=y^{(t)}$:

\begin{align} \frac{\partial \hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}} &=\frac{\partial \hat{\boldsymbol{y}}^{(t)}_i}{\partial o_i^{(t)}}\\ &=\frac{\exp(o_i^{(t)})\cdot\sum_j \exp(o_j^{(t)})-\exp(o_i^{(t)})\cdot\exp(o_i^{(t)})}{(\sum_j \exp(o_j^{(t)}))^2}\\ &=\frac{\exp(o_i^{(t)})}{\sum_j \exp(o_j^{(t)})} - (\frac{\exp(o_i^{(t)})}{\sum_j \exp(o_j^{(t)})})^2\\ &=\hat{\boldsymbol{y}}^{(t)}_i - (\hat{\boldsymbol{y}}^{(t)}_i)^2 \end{align} So \begin{align} \frac{\partial \log\hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}} &=\frac{1}{\hat{\boldsymbol{y}}^{(t)}_i}\cdot \frac{\partial \hat{\boldsymbol{y}}^{(t)}_i}{\partial o_i^{(t)}}\\ & = \frac{1}{\hat{\boldsymbol{y}}^{(t)}_i}\cdot (\hat{\boldsymbol{y}}^{(t)}_i - (\hat{\boldsymbol{y}}^{(t)}_i)^2)\\ &=1 - \hat{\boldsymbol{y}}^{(t)}_i \end{align}

When $i\ne y^{(t)}$:

\begin{align} \frac{\partial \hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}} &=\frac{-\exp(o_i^{(t)})\cdot\exp(o_{y^{(t)}}^{(t)})} {(\sum_j \exp(o_j^{(t)}))^2}\\ &=- \frac{\exp(o_i^{(t)})}{\sum_j \exp(o_j^{(t)})}\cdot \frac{\exp(o_{y^{(t)}}^{(t)})}{\sum_j \exp(o_j^{(t)})}\\ &= - \hat{\boldsymbol{y}}^{(t)}_i\cdot \hat{\boldsymbol{y}}^{(t)}_{y^{(t)}} \end{align} So \begin{align} \frac{\partial \log\hat{\boldsymbol{y}}^{(t)}_{y^{(t)}}}{\partial o_i^{(t)}} &=\frac{1}{\hat{\boldsymbol{y}}^{(t)}_i}\cdot \frac{\partial \hat{\boldsymbol{y}}^{(t)}_i}{\partial o_i^{(t)}}\\ & = \frac{1}{\hat{\boldsymbol{y}}^{(t)}_i}\cdot (- \hat{\boldsymbol{y}}^{(t)}_i\cdot \hat{\boldsymbol{y}}^{(t)}_{y^{(t)}})\\ &=- \hat{\boldsymbol{y}}^{(t)}_i \end{align}

So substitute into (2), we got : \begin{align} (\nabla_{o(t)}L)_i &= \begin{cases} \hat{\boldsymbol{y}}^{(t)}_i-1 &\text{If } i=y^{(t)},\\ \hat{\boldsymbol{y}}^{(t)}_i &\text{If } i\ne y^{(t)} \end{cases}\\ &=\hat{y}_i^{(t)} - \mathbf{1}_{i=y^{(t)}}. \qquad\qquad \text{which is (10.18)} \end{align}

So (10.18) proved.