0
$\begingroup$

In the word2vec paper (https://arxiv.org/pdf/1310.4546.pdf) that introduces the skip-gram algorithm we encounter this phrase: enter image description here which says that we maximize the average log probability.

Can someone help me understand how this formula maximizes? I don't have the intuition of what maximizing the probability means and why doing a sum over it would maximize it.

The only way I can think of maximizing is in the context of maximizing likelihood which you calculate using a probability density function.

$\endgroup$
4
  • 2
    $\begingroup$ This formula does not mazimize anything. You have to maximize it, usually with gradient descent. $\endgroup$
    – noe
    Oct 27, 2023 at 12:00
  • $\begingroup$ @noe thanks, I think I get it now. $\endgroup$ Oct 28, 2023 at 14:22
  • $\begingroup$ I have added my comment as an answer so your question can be marked as answered. $\endgroup$
    – noe
    Oct 28, 2023 at 14:44
  • $\begingroup$ This is maximum likelihood just with a particular structure in the joint probability density function based on how skip gram tries to model each word c positions around the current word $\endgroup$ Oct 30, 2023 at 22:22

1 Answer 1

1
$\begingroup$

This formula does not mazimize anything. You have to maximize it, usually with gradient descent.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.