I recently learned that there is a benchmark called NQ.


Unlike other QA benchmarks which relevant document is povided with query, it has to find information from millions of corpus by itself.

For example, if question is "when are hops added to the brewing process?" other QA benchmark also provide only 1 document about brewing. While NQ provide whole wikipedia text and model has to find most relevant document and answer.

When I tried all the example in the NQ with google search engine it gave me the answer every time. Then

  1. How does google search engine is managing those questions so well(Since NLP such as BERT is pretty computationally expensive I think it is not likely google is running whole model for every search)
  2. If google is doing it in other ways than neural network.(Or mixing it with other method) Do we need to bother with method such as REALM(https://arxiv.org/pdf/2002.08909.pdf) which seems to be computationally heavy?
  • $\begingroup$ Welcome to DataScienceSE. The problem with your question is that Google is a private company and the exact method they use is not public, as far as I'm aware. $\endgroup$
    – Erwan
    Commented Feb 22, 2021 at 12:06


Browse other questions tagged or ask your own question.