1
$\begingroup$

I have a dataset of 2000 documents where avg doc size is 300 words. The vocab is dominated by domain-specific words.

My goal is to find similar documents. For this, I tried LDA, LSI, Doc2Vec (topics=100) but results are not great. LSI is better than the others in my dataset. I also tried word2vec (size=100) & word movers distance but again no luck. I am thinking of trying POS tags & then building some ontology model but not sure.

Are my results poor because of a bad dataset? What are some other techniques that I can try?

$\endgroup$
1
  • 1
    $\begingroup$ Just a gut feeling from me, but 100 topics sounds like a lot with only 2000 docs. Where does 100 come from? Did you try fewer topics? $\endgroup$
    – kbrose
    Nov 1, 2017 at 18:18

3 Answers 3

1
$\begingroup$

Similarity of documents can be done with varied approaches.

As your documents are based on domain based words, you could employ a tfidf representation for each document and compute similarity based on this.

Previously, I have used word2vec representations of words and constructed document vectors by taking an average of all the vectors ( as one of the approaches ) and did a cosine similarity between these vectors. Basically an n*n similarity computation.

When you do topic modelling, even though the idea is to find documents which have similar topic distribution, there is a prior to the approach i.e identifying number of topics. Just using randomly 100 topics without looking at the distribution of the words might lead you on a wrong path.

A very interesting approach I have recently come across is combining topic modelling and word vectors, here is the link to a blog by stitchfix : http://multithreaded.stitchfix.com/blog/2016/05/27/lda2vec/

$\endgroup$
0
$\begingroup$

If you are unsure of determining number of topics which has to be passes as a piori. You could use coherence techniques either C_V or u_mass to get optimal number of topics for the data set.

$\endgroup$
0
0
$\begingroup$

One reason for the poor word embeddings could be the small number of documents. I think it's worth trying pre-trained word vectors (e.g. google news vectors: https://github.com/mmihaltz/word2vec-GoogleNews-vectors). You do mention a large number of domain-specific words, which could be an issue, but these vectors are trained with a very large vocabulary.

Otherwise I would second the idea of using something simpler like tf-idf scaled word frequencies, maybe limiting the vocabulary and reducing dimensionality.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.