Questions tagged [information-retrieval]
Information Retrieval is an area of study concerning with retrieving documents, information or metadata from a collection of unstructured or semi-structured data.
5
questions
10
votes
5
answers
18k
views
How to create a good list of stopwords
I am looking for some hints on how to curate a list of stopwords. Does someone know / can someone recommend a good method to extract stopword lists from the dataset itself for preprocessing and ...
- 333
6
votes
1
answer
107
views
Evaluating the performance of a machine learned recommendation system
I have a set of resumes $R=\{{r_1,...,r_n\}}$, which I've transformed to a vector space using TF-IDF. Each resume has a label, which is the name of their current employer. Each of these labels comes ...
- 467
4
votes
2
answers
3k
views
Document similarity: Vector embedding versus BoW performance?
I have a collection of documents, where each document is rapidly growing with time. The task is to find similar documents at any fixed time. I have two potential approaches:
A vector embedding (...
- 149
3
votes
1
answer
625
views
What is NLP technique to generalize manually created rules in text?
Let's say we have a free text containing key-value entities.
Example: "... patient's tumour has width 6 cm and height 5 cm"
Then an expert comes, marks it as important, thus we do have the rule for ...
- 133
1
vote
1
answer
7k
views
Cosine similarity between query and document confusion
I am going through the Manning book for Information retrieval. Currently I am at the part about cosine similarity. One thing is not clear for me.
Let's say that I have the tf idf vectors for the ...
- 123