I am working on a topic modeling for tweets projects. I have generated my topics using both unigrams and bigrams. Topics are defined with a mixture of both bigrams and unigrams.
Now I am planning to evaluate the coherence of my generated topics with the coherence measure I am trying to use is the sum of pair-wise similarity of terms that define the topics.
To compute that measure as you may see I need embeddings for my terms, and that is where the issue came from.
I have trained Fasttext on my corpus to learn the word embedding but it only gives me the embedding for unigrams and not bigrams.
My first question is how to train the embedding to learn both bigrams and unigrams embedding?
I found some research where they include n-grams to improve word embedding but I can't see any approach where they output embedding for bi_grams.
How should I process my text so that I can learn the embdeedings?
I found the Phrase and Phraser classes from Gensim but they are not returning all possibles embedding and I can't seem to understand how they are doing it.
Which another approach should I use?
- Can I split each sentence in my sentence to included both unigrams and bigrams and learn the embedding from them?
Ex: These are sentences about words embeddings === should be split into :
these, these_are, are, are_sentence, sentence, sentence_about, about, about_words, words words_enmbeddigns and learn the embedding from that?
- Each sentence can be split into one for unigrams and another one for bigrams and combine both and train the model on the combination? to
Ex: These are sentences about words embeddings should be split into :
these, are, sentences, about, words, embeddings and
these_are, are_sentences, sentences_about, about_words ,words_embeddings
Or simpler why not considering the bigrams embedding as the average of the unigrams embeddings?
All ideas on how to tackle this are welcomed...
Thanks
