I was reading the paper neural_approach_conversational_ai.pdf. And in the section Seq2Seq for Text Generation there is a formula that i feel a bit wrong [1]: https://i.stack.imgur.com/sX0it.png Can someone help me confirm this formula?
1
-
1$\begingroup$ That formula is not correct. Just check any negative log likelihood and you will see they differ. $\endgroup$– noeApr 7, 2022 at 9:24
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
This is the loss function that you aim to minimize by tuning the parameters theta given the data x,y. the loss is actually the negative conditional log-likelihood of the output sequence y given the input sequence x. what you want to find is a distribution P(y|x) parametrized by theta that gives you the most probable output sequence y given an input sequence x. minimizing the loss function means that you shape the distribution based on the examples in your training data such that for every sequence x in the training data the most probable output sequence y_predict agrees best with the actually observed output sequence y. You do this in the hope that the model will generalize well on unseen data, i.e. when you feed in a new sequence x that the model hasn't seen before, it will give you an accurate estimate of the corresponding sequence y that most likely will be associated with x.
-
$\begingroup$ The sign is not placed correctly and the indices and conditioning are not correct either. $\endgroup$– noeApr 7, 2022 at 9:24