Skip to main content
Share Your Experience: Take the 2024 Developer Survey
5 votes

Why do popular search engines not follow the usual AND, OR logic for queries?

Nice question! An exact answer should be given by looking in the search engine source code but here is a possible explanation. I run the queries at Google burglar 33,800,000 burglar AND burglar 29,...
DaL's user avatar
  • 2,643
4 votes

Why do popular search engines not follow the usual AND, OR logic for queries?

Google used to do, to some extend. For a long time, using +word could be used to require the presence of a word. So "a AND b" would be "+a +b" whereas "a OR b" would be "a b" (with a preference to ...
Has QUIT--Anony-Mousse's user avatar
3 votes
Accepted

Finding similar articles in realtime

Elasticsearch is the right tool to use if you don't want to code this yourself. Indeed, you need an indexing algorithm that is able to efficiently retrieve pieces of texts in a big database, and SQL ...
Robin's user avatar
  • 1,337
2 votes

Best method for similarity searching on 10,000 data points with 8,000 features each in Python?

There are two main paths: Load all vectors into memory. If you are able to load vectors into memory, then you might be able to search the space with "clever" brute force. One such method is found in ...
Brian Spiering's user avatar
2 votes
Accepted

What is the difference between Okapi bm25 and NMSLIB?

Note that I don't know nmslib and I'm not familiar with search optimization in general. However I know Okapi BM25 weighting. How do they both (bm25, nmslib) differ? These are two completely ...
Erwan's user avatar
  • 25.5k
2 votes
Accepted

scalable tools to build kNN graph over sparse data

I think what you might be looking for is, L2Knng: Fast Exact K-Nearest Neighbor Graph Construction with L2-Norm Pruning They have multiple runtime options specifically for different kinds of ...
Sujay_K's user avatar
  • 111
2 votes

What ML/DL techniques power Youtube/Netflix search systems?

At Netflix, the machine ​learning algorithms used are more complex since the data is really vast and increases day to day. Take a look at these: Netflix Recommendations: Beyond the 5 stars (Part 1) ...
Abhishek Sharma's user avatar
2 votes

How does Google's 'showing results for' work?

First of all, no one knows how google search works except what google officially publishes. But I give you a simple algorithm for query correction (I have implemented this previously in production). ...
Kasra Manshaei's user avatar
2 votes

Best way to vectorise names and addresses for similarity searching?

The problem you are describing is commonly called record linkage, in particular probabilistic record linkage. Clustering the embeddings would work if the different personas for the same entity ...
Brian Spiering's user avatar
2 votes
Accepted

Purely extractive Language Model

You can prepend each line of the email with a line number and request the LLM to give you the initial and final line numbers of the most recent email, separated by "-". Then, you can parse ...
noe's user avatar
  • 26.9k
1 vote

Semantic Search on numeric data

You could feed the LLM a description of the file format and then request it to generate a piece of code to extract the information you want, for instance, in Python. Then, you would run the generated ...
noe's user avatar
  • 26.9k
1 vote
Accepted

How do I use FAISS searching with Haystack?

DocumentStore As you can read in the docs, you can think of the DocumentStore as a database that stores your texts and meta data and provides them to the Retriever at query time. FAISSDocumentStore <...
Stefano Fiorucci - anakin87's user avatar
1 vote

Learning to Rank with Unlabelled Dataset

There are different ways to look at this: You can apply a totally unsupervised method, like computing a TDIDF vector for the query and then ranking according to its similarity (e.g. cosine) against ...
Erwan's user avatar
  • 25.5k
1 vote

Why do we calculate the vector of a document by averaging the vectors of all the words?

we should obtain the vector of a document by averaging the vectors of all the words This is not necessarily the case. But surely it is a convenient approach. The main advantage in particular, is to ...
Edoardo Guerriero's user avatar
1 vote
Accepted

Could you generate search queries to poison data analysis by a search engine?

I am not sure how many queries you'd need to perform to drown out your actual search queries, but there is already is an actual browser addon which does this. This addon is called TrackMeNot and is ...
Oxbowerce's user avatar
  • 7,592
1 vote

What is the formula and log base for idf?

There a a number of variation how to calculate inverse document frequency. Have a look at the wiki page (Tf-Idf) or scikit-learn's TfidfVetorizer class.
Tinu's user avatar
  • 508
1 vote

Measuring quality of answers from QnA systems

The ranking of the answers is part of the ML process, i.e. a system should be trained to rank the answers according to their relevance. Heuristic measures such as the ones mentioned in your question ...
Erwan's user avatar
  • 25.5k
1 vote

How can I improve the recall of a certain class in a multiclass-classification result

A couple options: Increase the confidence score corresponding to your class-of-interest until you reach the desired recall Upsample the class you wish to have better recall on in the training set ...
Michael Higgins's user avatar
1 vote

How can I improve the recall of a certain class in a multiclass-classification result

Recall is the ability of a search model to find the correctly labeled items amongst all the items for a given query. One common method to improve specific query results is to create a custom model. ...
Brian Spiering's user avatar
1 vote

Is Elastic Search recommended if attribute getting search is not a huge text document?

As you mentioned you are searching just names, emails or ids, etc which is not large text. So consider a case where you have 6 documents/records having names as below then you can understand better ...
Aman Tandon's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible