1
$\begingroup$

For my site I'm working on a chat recommender that would recommend chats to users. Each chat has a title and description and my corpus is composed of many of these title and description documents. I was curious about training an LDA2Vec model, but considering the fact that this is a very dynamic corpus that would be changing on a minute by minute basis, it's not doable. I was thinking of just doing standard LDA, because LDA being a probabilistic model, it doesn't require any training, at the cost of not leveraging local inter-word relationships. The other added benefit of LDA2Vec was that I could get accurate labeled topics. So I thought, what if I use standard LDA to generate the topics, but then I use a pre-trained word2vec model whether that be trained locally on my corpus or a global one, maybe there's a way to combine both.

The junk below draws heavily from the stuff in the lda2vec paper:

https://arxiv.org/pdf/1605.02019.pdf

topic 0: SpaceX, Nasa, Asteroids, Rover

w_j = target word

w_i = pivot word

d_j = document vector or what I'm calling the topic in my scenario

c_j = context vector

c_j = w_j + d_j

d_j = [v_1, v_2 ... v_n]

This is the loss function from the paper

Loss Function

I can then iterate each word w_j for each pivot w_i such that the loss is minimal:

so first iteration pivot word is w_i = SpaceX and w_j iterates over [Nasa, Asteroids, Rover]

second iteration pivot word is w_i = Nasa and w_j iterates over [SpaceX, Asteroids, Rover]

once it's done I'll have a document vector d_j, I'll find the closest word in the embedding space to d_j and I presume that's a good label for the topic. Either I'm being extremely naive or this maybe works. Any input would be much appreciated, thanks in advance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.