I'm trying to understand https://storage.googleapis.com/pub-tools-public-publication-data/pdf/45530.pdf
Their candidate generation step outputs top N items
- via softmax (with negative sampling) at training time .
- via nearestneighbor at serving time.
I guess $v_j$ represents, (from softmax layer to nearest neighbor index)
topn videos you get via softmax, and represent them in the original encoding (same encoding you used for the input (used for embedded video watches))
apparently, $v_j$ are in the different encoding from the input encodings.
The softmax layer outputs a multinomial distribution over the same 1M video classes with a dimension of 256 (which can be thought of as a separate output video embedding)
I'm trying to understand what they mean by interpreting softmax output as a separate output video embedding. I thought softmax layer that outputs 1M classes has dimension of 1M, where does 256 came from? (It's the same question as How to create a multi-dimensional softmax output in Tensorflow? and I don't think it has been answered there..)
user vector $u$ is the output of the final ReLU unit, although I'm not sure what this user vector is used for.
I guess in serving time, to pick the top N for a given user, user vector $u$ is used by nearest-neighbor. But my understanding of nearest-neighbor is for a given vector, it finds nearest vectors in the same dimension. (such as given an movie, find nearest movies). However here, you are given a user and need to find topn videos. How does that work?
My best guess is that, for a given user, u get a user vector as the ReLU output, then find user-user nearest neighbor, and combine their topn items obtained in the training time. But it's just a guess..