How PV-DBOW works

Question

The authors of the Paragraph Vector paper describe PV-DBOW with:

2.3. Paragraph Vector without word ordering: Distributed bag of words

The above method considers the concatenation of the paragraph vector with the word vectors to predict the next word in a text window. Another way is to ignore the context words in the input, but force the model to predict words randomly sampled from the paragraph in the output. In reality, what this means is that at each iteration of stochastic gradient descent, we sample a text window, then sample a random word from the text window and form a classification task given the Paragraph Vector.

I have a couple of questions:

1. Why do you need to sample a text window before sampling a random word? To create a batch, why can't you just randomly sample from a list of the form [(1, "cat"), (1, "sat"), ..., (1, "mat"), (2, "humpty"), (2, "dumpty"), ... (2, "wall"), ...] where the first item in each tuple represents the paragraph?
2. If hierarchical softmax or negative sampling is used, is stochastic gradient descent still used to update the weights in the network? Or are these optimization methods themselves?
3. To infer representations for new paragraphs, is the model only trained on words sampled from that paragraph?

Answer 1

1. The paragraph vectors are trained by using the information of words in a paragraph. If we randomly sample without distinguishing paragraph as you suggest, a paragraph vector will be adjusted in vector space by other words which are not components of the paragraph. So it will map/project the paragraph vector by irrelevant syntactical and semantical information of words.

2. Both hierarchical softmax and negative sampling are not one of gradient-based optimizer. These methods just adjust/change the objective function to train much easier and faster.

3. I'm not sure but I think it’s not possible. Check this: doc2vec - How does the inference step work in PV-DBOW