I am now taking Andrew Ng's deep learning course on Coursera. Everything is great but when it comes to RNN, I sometimes feel confused. Here is a question about RNN (or more specifically, the RNN loss).

To my understanding , in the following snapshot, the $i$-th example is a tuple that looks like $$(\mathbf{x}^{(i)}, \mathbf{y}^{(i)}):=(\{\mathbf{x}^{(i)<1>},\cdots, \mathbf{x}^{(i)<T_x^{(i)}>}\}, \{\mathbf{y}^{(i)<1>},\cdots, \mathbf{y}^{(i)<T_y^{(i)}>}\})$$ where $T_x^{<i>}$ and $T_y^{<i>}$ are not necessarily equal. enter image description here The $\mathbf{x}^{(i)<t>}$'s are fed into RNN units and we get prediction $\hat{\mathbf{y}}^{(i)<t>}$'s. Suppose there are $m$ examples in total, the loss should actually be $$ \ell = \sum_{i=1}^m\sum_{t=1}^{T_y^{(i)}} \ell(\hat{\mathbf{y}}^{(i)<t>}, \mathbf{y}^{(i)<t>}) $$ which seems not to be consistent with Andrew's notation. In his notation, the loss of each individual RNN unit seems to be different (therefore he has $\ell^{<t>}(\cdot)$ instead of generic $\ell(\cdot)$).

I am wondering if my understanding is correct. Could anyone help me. Thank you in advance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.