Linked Questions
11 questions linked to/from Unsupervised document similarity state of the art
15
votes
4
answers
13k
views
Alternatives to TF-IDF and Cosine Similarity when comparing documents of differing formats
I've been working on a small, personal project which takes a user's job skills and suggests the most ideal career for them based on those skills. I use a database of job listings to achieve this. At ...
- 151
4
votes
3
answers
11k
views
How to measure the similarity between two text documents?
Assume, I have 100 text documents, and I want to cluster those documents.
The first step is the construct pairwise similarity matrix 100X100 for the documents
My ...
- 319
6
votes
3
answers
6k
views
Weighted sum of word vectors for document similarity
I have trained a word2vec model on a corpus of documents. I then compute the term frequency (the same Tf in TfIDF) of each word in each document, multiply each words Tf by its corresponding word ...
- 1,594
1
vote
1
answer
7k
views
Cosine similarity between query and document confusion
I am going through the Manning book for Information retrieval. Currently I am at the part about cosine similarity. One thing is not clear for me.
Let's say that I have the tf idf vectors for the ...
- 123
3
votes
2
answers
1k
views
Cluster documents based on topic similarity
I have set of documents where I have assigned topics per each document.
E.g., Topics of document 1 -> 1.0 Science, 1.0 politics, 0.8 History, 0. 8 Information and Technology
Now I want to cluster ...
- 529
4
votes
2
answers
3k
views
Document similarity: Vector embedding versus BoW performance?
I have a collection of documents, where each document is rapidly growing with time. The task is to find similar documents at any fixed time. I have two potential approaches:
A vector embedding (...
- 149
1
vote
1
answer
2k
views
Use embeddings to find similarity between documents
I need to find cosine similarity between two text documents. I need embeddings that reflect order of the word sequence, so I don't plan to use document vectors built with bag of words or TF/IDF. ...
- 295
2
votes
2
answers
345
views
Document similarity
I have close to 50000 documents in plain text format.
Is there a way in which I can group similar documents together? Similarity mostly here is the content similarity.
Will transforming the text ...
- 149
1
vote
2
answers
434
views
Word2Vec - document similarity
Lets say I have text data for different documents from 2005 - 2015. I want to compare the similarity between $t$ and $t-1$ documents. So I take the document at 2006 and compare it with the document at ...
- 111
1
vote
1
answer
343
views
What is considered short and long text in NLP (document similarity)
What is considered short and long text in NLP?
I'm working on a dataset that contains documents from 10 to 600 words and I'm asking myself if I should treat them differently. Also, I haven't found a ...
- 143
1
vote
2
answers
201
views
Evaluate document similarity / content-based recommender system
I'm planning on building a basic content-based recommender system with word2vec and cosine similarity.
The data consists of 300k documents in varying length.
How do I evaluate my model if I have no ...
- 143