12
$\begingroup$

I'm curious about natural language querying. Stanford has what looks to be a strong set of software for processing natural language. I've also seen the Apache OpenNLP library, and the General Architecture for Text Engineering.

There are an incredible amount of uses for natural language processing and that makes the documentation of these projects difficult to quickly absorb.

Can you simplify things for me a bit and at a high level outline the tasks necessary for performing a basic translation of simple questions into SQL?

The first rectangle on my flow chart is a bit of a mystery.

enter image description here

For example, I might want to know:

How many books were sold last month?

And I'd want that translated into

Select count(*) 
  from sales 
  where 
   item_type='book' and 
   sales_date >= '5/1/2014' and 
   sales_date <= '5/31/2014'
Improve this question
$\endgroup$

4 Answers 4

Reset to default
7
$\begingroup$

Natural language querying poses very many intricacies which can be very difficult to generalize. From a high level, I would start with trying to think of things in terms of nouns and verbs.

So for the sentence: How many books were sold last month?

You would start by breaking the sentence down with a parser which will return a tree format similar to this:

enter image description here

You can see that there is a subject books, a compound verbal phrase indicating the past action of sell, and then a noun phrase where you have the time focus of a month.

We can further break down the subject for modifiers: "how many" for books, and "last" for month.

Once you have broken the sentence down you need to map those elements to sql language e.g.: how many => count, books => book, sold => sales, month => sales_date (interval), and so on.

Finally, once you have the elements of the language you just need to come up with a set of rules for how different entities interact with each other, which leaves you with:

Select count(*) from sales where item_type='book' and sales_date >= '5/1/2014' and sales_date <= '5/31/2014'

This is at a high level how I would begin, while almost every step I have mentioned is non-trivial and really the rabbit hole can be endless, this should give you many of the dots to connect.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Any papers recommended for what you are saying? $\endgroup$
    – Albert Chen
    Feb 25, 2021 at 19:28
1
$\begingroup$

Turning simple questions into answers is not simple whatsoever.

The first technology to do this as broadly across technology and accurately will be a big winner.

However, there are some out there, filling in the gaps with "answering questions" with Artificial Intelligence (e.g. IBM Watson, and Amazon Alexa). This requires solving the language complexities related to the data in question, what's in the data stores, and what are nouns, verbs and pronouns.

Microsoft ventured here with English Query but, then stopped. Kueri.me is a Python based platform doing about the same thing.

Structured Query Languages (SQL) and the like, SOQL, MDX, Hive, Impala and the newer takes on old fashioned SQL. Have not yet replaced much of anything, all these pieces are small fixes to the grander "End Goal" and that lies in Artificial Intelligence (AI), specifically, Machine Learning.

The question being:

"Can the computer, figure out what you want."

Not yet. It takes Linguists, Mathematicians, Engineers and more to all contribute their piece of the pie so we can enjoy some of that sweet Artificial Intelligent and Machine Learned cake.

Improve this answer
$\endgroup$
0
$\begingroup$

There are several approaches to creating a parser that would parse plain text message into SQL. For example, you can create a grammar-based parser and use an NLP algorithm to build the structured query. If you already have plenty of parsed messages from one domain (like e-commerce) - you can try a Machine Learning approach and use it for your further parsing.

However, I think the best approach is to combine a grammar-based parser for text-to-SQL translation, and ML to complement the rule-based grammar by fixing the syntax, eradicating typos, etc.

Learn more about the different approaches here.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.