I was trying to build a simple negative sampler for a Word2Vec model using TensorFlow by following the tutorial here.
From what I understand, the
tf.random.log_uniform_candidate_sampler() takes as input the correct class to be exluded from sampling, the number of samples to return, and the size of the distribution to sample.
In the tutorial, the
true_classes is 1,
range_max is 8 and
num_ns is 4. Which translates to, "Return 4 samples from the distribution [0, 8), but skip 1, since it's a positive class."
However, in the tutorial, the sampler returns the following tensor:
tf.Tensor([2 1 4 3], shape=(4,), dtype=int64).
My question is, why does the sampler return the class 1, even after being told to exclude it, since it doesn't constitute a negative sample?
Furthermore, in the following section of the tutorial:
target_index : 7 target_word : sun context_indices : [1 2 1 4 3] context_words : ['the', 'wide', 'the', 'shimmered', 'road'] label : [1 0 0 0 0]
We can see clearly that the first instance of class 1 (index 0 in
context_indices) has been flagged as a positive skip-gram, whereas the second instance of 1 (index 2 in
context_indices) has been labelled as a negative skip-gram.
I'd love some clarification or a correction in my understanding of this model or tutorial.