For my site I'm working on a chat recommender that would recommend chats to users. Each chat has a title and description and my corpus is composed of many of these title and description documents. I was curious about training an LDA2Vec model, but considering the fact that this is a very dynamic corpus that would be changing on a minute by minute basis, it's not doable. I was thinking of just doing standard LDA, because LDA being a probabilistic model, it doesn't require any training, at the cost of not leveraging local inter-word relationships. The other added benefit of LDA2Vec was that I could get accurate labeled topics. So I thought, what if I use standard LDA to generate the topics, but then I use a pre-trained word2vec model whether that be trained locally on my corpus or a global one, maybe there's a way to combine both.
The junk below draws heavily from the stuff in the lda2vec paper:
topic 0: SpaceX, Nasa, Asteroids, Rover
w_j = target word
w_i = pivot word
d_j = document vector or what I'm calling the topic in my scenario
c_j = context vector
c_j = w_j + d_j
d_j = [v_1, v_2 ... v_n]
This is the loss function from the paper
I can then iterate each word w_j for each pivot w_i such that the loss is minimal:
so first iteration pivot word is w_i = SpaceX and w_j iterates over [Nasa, Asteroids, Rover]
second iteration pivot word is w_i = Nasa and w_j iterates over [SpaceX, Asteroids, Rover]
once it's done I'll have a document vector d_j, I'll find the closest word in the embedding space to d_j and I presume that's a good label for the topic. Either I'm being extremely naive or this maybe works. Any input would be much appreciated, thanks in advance.