Skip to main content
8 votes

How can you build a model that extracts data out from receipts?

The simplest pipeline would be to do the following: OCR Named Entity Extraction Entity Disambiguation OCR This is basically transforming your receipts into plain text. If you have scans (pictures) ...
Bruno Lubascher's user avatar
7 votes
Accepted

Difference between paragraph2vec and doc2vec

There may be differing implementations, but these two terms refer to the same thing. Both convert a generic block of text into a vector similarly to how word2vec converts a word to vector. Paragraph ...
jncraton's user avatar
  • 578
5 votes

Why do popular search engines not follow the usual AND, OR logic for queries?

Nice question! An exact answer should be given by looking in the search engine source code but here is a possible explanation. I run the queries at Google burglar 33,800,000 burglar AND burglar 29,...
DaL's user avatar
  • 2,643
4 votes

How can conclusions be drawn from recommendation systems evaluation?

"Good", I think, is based on the state of the art at the moment. So I would look at respected models from industry leaders and use their reported accuracies as a base line for what is "good": since ...
grldsndrs's user avatar
  • 567
4 votes

how to evaluate top n recommendation system with movie lens dataset?

For various metrics feel free to look at various benchmarking libraries including MyMediaLite and LibRec. If you are doing a TOP N approach, then the way to evaluate this using a Movielens system is ...
Skylion's user avatar
  • 141
4 votes

Why do popular search engines not follow the usual AND, OR logic for queries?

Google used to do, to some extend. For a long time, using +word could be used to require the presence of a word. So "a AND b" would be "+a +b" whereas "a OR b" would be "a b" (with a preference to ...
Has QUIT--Anony-Mousse's user avatar
4 votes
Accepted

How is "relevance" defined in information retrieval outside the context of systems with user feedback?

There is no formal definition for the concept of relevance, because relevance depends completely on the context and is therefore highly subjective. This is why the best way (some might say the only ...
Erwan's user avatar
  • 25.5k
3 votes
Accepted

Doc2Vec or Word2vec for word embedding

As the name implies, doc2vec generates vectors representing documents (sentences, paragraphs) but not single words. So training doc2vec won't give you word vectors but document vectors. This means you ...
z80crew's user avatar
  • 146
3 votes

I have 50 videos. I ask a customer 10 questions. Based on their answers, I send them a set of videos. How do I do it?

Frame this as a classification problem and learn a decision tree to map question responses to video selections. EDIT: Fleshing this out a bit more: Collect appropriate data. Get members of your ...
David Marx's user avatar
  • 3,258
3 votes
Accepted

Plotting Precision Recall Curve

In short, the precision-recall curve shows the trade-off between the two values as you change the strictness of the classifier. There is a great explanation here, using the classification of images ...
KT12's user avatar
  • 240
3 votes

How can conclusions be drawn from recommendation systems evaluation?

First of all, it's important to recall that RMSE has the same unit as the dependent variable (DV). It means that there is no absolute good or bad threshold. However, you can define it based on your DV....
Abhishek's user avatar
  • 1,989
3 votes
Accepted

Can macro F1 score be greater than micro F1 score?

Yes, it can. Example with Precision Class A: 1 TP and 1 FP Class B: 10 TP and 90 FP Class C: 1 TP and 1 FP Class D: 1 TP and 1 FP Here, $P_A = P_C = P_D = 0.5$, $P_B = 0.1$ Macro-F1 is: $P_M = \...
Antonio Jurić's user avatar
3 votes

How "similarity" is measured in image retrieval?

There is no single best similarity metric, unless a query and found images are near identical. Similarity is not a universal concept. It is trained. Maybe it happened to you to say that a person A ...
Similar pictures's user avatar
2 votes

How can conclusions be drawn from recommendation systems evaluation?

You've already mentioned a few metrics and I guess they work, at least from a algorithmic point of view. However, I think a lot of the initial validation would really has to be done manually. I work ...
Nanda's user avatar
  • 773
2 votes

Plotting Precision Recall Curve

What is the x and y axis of this scatter plot? If precision and recall are on these axes, then the range of both axes would be 0 - 1. I'm assuming one point would then represent one model, in this ...
jkyh's user avatar
  • 462
2 votes
Accepted

Text Mining of Research Paper Abstracts

In order to train some supervised learning algorithm to identify 'Problem' and 'Solution', you need to somehow generate some data that has labels of these things, which may be your best bet. So you ...
CHP's user avatar
  • 170
2 votes

Learning to rank: construct absolute ranking using pair-wise ranking approach

Creating a total ranking from pairwise comparisons that don’t necessarily follow the axioms or rational preferences would certainly require some optimization, and you would need to compute a quadratic ...
anymous.asker's user avatar
2 votes

Document similarity: Vector embedding versus BoW performance?

Here some points on which we can focus - 1.) Averaging the words vectors lose the order of words, making it very similar to the concept of Bag of Words. That is why with less Data Bag of words ...
Abhishek Verma's user avatar
2 votes
Accepted

Does recall has different interpretation when comes to classification and information retrieval

According to wikipedia, Recall is defined as- In information retrieval, recall is the fraction of the relevant documents that are successfully retrieved. In your second formula, recall = p/q, where ...
Ankit Seth's user avatar
  • 1,821
2 votes

Industrial application(s) of LDA (latent Dirichlet allocation)?

I can't answer for the systems in already existing companies, but I can definitely share an application of LDA in NLP. Latent Dirichlet Allocation is a popular technique use for topic modelling in ...
Kaustubh's user avatar
  • 146
2 votes

Two definitions of DCG measure

I believe you are correct, that the paper and Wikipedia disagree. The paper's formula suggests you apply no discount at $r <= b$, which means both of the first two elements are not discounted. The ...
Sean Owen's user avatar
  • 6,595
2 votes

Algorithm for document retrieval in QA system

I am still not quite sure how you are solving in the problem with tf-idf for a QA system. However, there have been lots of improvements that has been done in the QA domain over the years, with the ...
Nischal Hp's user avatar
2 votes
Accepted

Word embeddings for Information Retrieval - Document search?

Doc2Vec is on possible approach. With this, model learns to "cluster" similar sentences together. Most simplistic approach is to aggregate word vectors but that ignores order of words. Details on ...
Shamit Verma's user avatar
  • 2,259
2 votes

Best way to combine two similar document

Your description could corresponds to the NLP task of summarization. This is an active field of research: https://scholar.google.com/scholar?q=text+summarization A much simpler option is to only ...
Erwan's user avatar
  • 25.5k
2 votes
Accepted

Intuition behind the entropy definition

Pls. refer Pattern recognition and Machine Learning for details - section 1.6: Information Theory. Left graph is Information(events) vs Probability. And the right graph is Expected Information/(...
Continue2Learn's user avatar
2 votes
Accepted

Why TREC set two task: document ranking and passage ranking

The idea behind the two tasks is to explore how document length affects the effectiveness of the different retrieval models. Ellen Voorhees TREC project manager NIST
Ellen Voorhees's user avatar
2 votes
Accepted

TS-SS and Cosine similarity among text documents using TF-IDF in Python

Normalised vectors have magnitude 1, so it doesn't matter if you explicitly divide by the magnitudes or not. It's mathematically equivalent either way. I see no reason that you couldn't use ...
timleathart's user avatar
  • 3,940
2 votes
Accepted

Evaluating a IR system (Precision and Recall)

My doubt is related to the following question: Do we use those two measures only if we are testing a IR system or not? Technically the answer is no because precision and recall are used to evaluate ...
Erwan's user avatar
  • 25.5k
2 votes
Accepted

Building a tag-based recommendation engine given a set of user tags?

If I got your problem description correct, you are looking for a recommender system like for example used by Netflix or Amazon. State of the art solution would be to use Latent Dirichlet Allocation ...
Eugen's user avatar
  • 457
2 votes
Accepted

What is the difference between Okapi bm25 and NMSLIB?

Note that I don't know nmslib and I'm not familiar with search optimization in general. However I know Okapi BM25 weighting. How do they both (bm25, nmslib) differ? These are two completely ...
Erwan's user avatar
  • 25.5k

Only top scored, non community-wiki answers of a minimum length are eligible