2
$\begingroup$

I have documents, say for example, D1, D2, D3... Dm.

Every Di has its individual components or keywords k1, k2, k3,... kn, where ki is an n-dimensional vector. The number of individual components varies between documents.

What are the ways to find how close Di are? Or what is the best way to represent a document using its keyword vectors? Please note that I'm using a custom embedding here.

| improve this question | |
$\endgroup$
  • 1
    $\begingroup$ Can you specify what you mean by "how close Di are"? Are you looking for semantic similarity or something else? Also, how to "represent a document" depends on what goal you're trying to achieve, which again is not clear. $\endgroup$ – Alex L Mar 17 '19 at 3:40
  • $\begingroup$ @AlexL thanks for the comment. I'm trying to find similarity between documents using its keywords. By closeness I meant how similar two or more documents are. In order to find similarity between two documents, I need to find a way to represent the documents using its individual keyword vectors I obtained from an embedding technique, say for example, tf-idf. I hope this clarifies. $\endgroup$ – Van Peer Mar 17 '19 at 9:04
  • 1
    $\begingroup$ You've just defined 'closeness' in terms of 'similarity', which is undefined here. It's common to define similarity in terms of cosine similarity of the doc vectors; if that's not what you mean, clarify further. $\endgroup$ – Sean Owen Mar 17 '19 at 23:48
3
$\begingroup$

If you have the vectors for your keywords, you can aggregate those to get the document vectors. The simplest(and probably one of the most effective) way is to average out your keyword vectors to form the document vector. Once you have the vectors for each of the documents, you can use similarity measures like cosine similarity to see how close your documents are. A typical example of such a method is this : Source: https://www.slideshare.net/mobile/andrewkoo/word2vec-algorithm

Edit: Another interesting point would be to evaluate how you are getting your word vectors. Please look at pretrained word embeddings like word2vec, Glove or Fast Text.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Do you have any references that links to averaging? What other options do I've apart from averaging? I've looked into pre-trained word embeddings as well. Are you suggesting to use averaging for that as well? Thanks! $\endgroup$ – Van Peer Mar 17 '19 at 18:27
2
$\begingroup$

It sounds like you're trying to do some sort of topic modeling. I might recommend something like Latent Semantic Analysis (LSA), Latent Dirichlet allocation (LDA), or Latent Semantic Indexing (LSI). LSA performs a matrix decomposition of the document x keyword occurrence matrix to extract salient topics. It can allow you to find the cosine similarity between documents, just as you're looking for. I recommend this introduction paper. As this SO answer suggests though, tf-idf can also be used as a document vector if that's what you wish to use.

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ LSA is time-tested and has been applied to many areas of research, so if I were you I'd take a good look at that. If nothing here is what you're looking for, there are more options too. Googling any of these ideas should present you with many tutorials and guides! $\endgroup$ – Alex L Mar 17 '19 at 15:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.