1
$\begingroup$

I was reading the paper neural_approach_conversational_ai.pdf. And in the section Seq2Seq for Text Generation there is a formula that i feel a bit wrong [1]: https://i.stack.imgur.com/sX0it.png Can someone help me confirm this formula?

$\endgroup$
1
  • 2
    $\begingroup$ That formula is not correct. Just check any negative log likelihood and you will see they differ. $\endgroup$
    – noe
    Apr 7, 2022 at 9:24

1 Answer 1

0
$\begingroup$

This is the loss function that you aim to minimize by tuning the parameters theta given the data x,y. the loss is actually the negative conditional log-likelihood of the output sequence y given the input sequence x. what you want to find is a distribution P(y|x) parametrized by theta that gives you the most probable output sequence y given an input sequence x. minimizing the loss function means that you shape the distribution based on the examples in your training data such that for every sequence x in the training data the most probable output sequence y_predict agrees best with the actually observed output sequence y. You do this in the hope that the model will generalize well on unseen data, i.e. when you feed in a new sequence x that the model hasn't seen before, it will give you an accurate estimate of the corresponding sequence y that most likely will be associated with x.

$\endgroup$
1
  • 1
    $\begingroup$ The sign is not placed correctly and the indices and conditioning are not correct either. $\endgroup$
    – noe
    Apr 7, 2022 at 9:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.