# Word2Vec, softmax function

I was going term by term through the softmax function for the word2vec (SKIP-GRAM) model. I found most definition of these functions to be not 'clear' so I modified the notation to make sure I understand it.

Is the following formulation correct?

$$P(w_{-t} | w_{t} ; \theta) = softmax(score(w_{-t}, w_t))$$

$$P(w_{-t} | w_{t} ; \theta) = \frac{exp(score(w_{-t}, w_t))}{\sum_{w' \in \theta} exp(score(w', w_{t}))}$$

where:

$w_{-t} =$ context

$w_{t} =$ target word

$score(A,B)$ a measure of similarity between vectors A and B.

$\theta =$ vector representation for all words in vocabulary

In the simplest case:

$$score(A ,B) = A \cdot B$$

\begin{align} P(w_t | h) &= \text{softmax} (\text{score} (w_t, h)) \\ \\ &= \frac{\exp \{ \text{score} (w_t, h) \} } {\sum_\text{Word w' in Vocab} \exp \{ \text{score} (w', h) \}} \end{align}
It's the same as yours, but they generalize the condition $h$ as a history.
In case of skip-gram, the target word $w_t$ is any context word ($w_{-t}$ in your notation) and the history is the center word ($w_{t}$ in your notation). You specify $\theta$ explicitly in the condition, but that is usually omitted, because there is only one vocabulary in a given problem.