0
$\begingroup$

Problem

I am now taking Andrew Ng's deep learning course on Coursera. Everything is great but when it comes to RNN, I sometimes feel confused. Here is a question about RNN (or more specifically, the RNN loss).

To my understanding , in the following snapshot, the $i$-th example is a tuple that looks like $$(\mathbf{x}^{(i)}, \mathbf{y}^{(i)}):=(\{\mathbf{x}^{(i)<1>},\cdots, \mathbf{x}^{(i)<T_x^{(i)}>}\}, \{\mathbf{y}^{(i)<1>},\cdots, \mathbf{y}^{(i)<T_y^{(i)}>}\})$$ where $T_x^{<i>}$ and $T_y^{<i>}$ are not necessarily equal. enter image description here The $\mathbf{x}^{(i)<t>}$'s are fed into RNN units and we get prediction $\hat{\mathbf{y}}^{(i)<t>}$'s. Suppose there are $m$ examples in total, the loss should actually be $$ \ell = \sum_{i=1}^m\sum_{t=1}^{T_y^{(i)}} \ell(\hat{\mathbf{y}}^{(i)<t>}, \mathbf{y}^{(i)<t>}) $$ which seems not to be consistent with Andrew's notation. In his notation, the loss of each individual RNN unit seems to be different (therefore he has $\ell^{<t>}(\cdot)$ instead of generic $\ell(\cdot)$).

I am wondering if my understanding is correct. Could anyone help me. Thank you in advance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.