26
$\begingroup$

I have been working on developing a system "Converting Natural Language to SQL Query".

I have read the answers from the similar questions, but was not able to get the information that I was looking for.

Below is the flowchart for such system which I have got from An Algorithm to Transform Natural Language into SQL Queries for Relational Databases by Garima Singh, Arun Solanki

Flowchart

I have understood till part of speech tagging step. But how do I approach the remaining steps.

  1. Do I need to train all the possible SQL queries?
  2. Or, once part of speech tagging is done, I have to play with the words and form a SQL query?

Edit: I have successfully implemented the from step "user query" to "Part of speech tagging".

Thank you.

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ At Nibi.ai (I am one of the founders) we are building an NLP to SQL engine that you can use as an API. We are launching soon. Let me know if you want to get a demo. $\endgroup$
    – Yehuda Kogan
    Aug 29, 2018 at 13:55
  • 1
    $\begingroup$ As an alternative you may ask the human to take a SQL course... $\endgroup$
    – Marmite Bomber
    Aug 29, 2018 at 22:54

5 Answers 5

Reset to default
21
+25
$\begingroup$

If you want to tackle the problem from another perspective, with an end to end learning, such that you don't specify ahead of time this large pipeline you've mentioned earlier, all you care about is the mapping between sentences and their corresponding SQL queries.

Tutorials:

How to talk to your database

Papers:

Dataset:

A large annotated semantic parsing corpus for developing natural language interfaces.

Github code:

  1. seq2sql
  2. SQLNet

Also, there are commercial solutions like nlsql

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ +1, for answering well but haven't gone through the links yet $\endgroup$
    – Toros91
    May 16, 2018 at 6:11
  • $\begingroup$ @Fadi Bakoura Thank you. Let me go through the links . $\endgroup$
    – deepguy
    May 16, 2018 at 6:56
6
$\begingroup$

NLTK has an excellent step by step guide on everything you need to convert human language to an SQL query using the nltk package in python.

It’s rudimentary, but it answers your question.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks @killerT2333 . I just had look. But it is kind of confusing. Is there any other simple doc ? $\endgroup$
    – deepguy
    May 14, 2018 at 16:15
  • 3
    $\begingroup$ That's the simplest one I know of - it's quite a complex task what you're asking, so there's no simple answer to your question. On the nltk documentation they do take you through the theory at a high level, and at also at a low level with a lot of code examples. More extensive than that, you probably need to search github or research papers. $\endgroup$
    – PyRsquared
    May 14, 2018 at 16:17
  • $\begingroup$ I will go through that one more time. And update you here. $\endgroup$
    – deepguy
    May 14, 2018 at 16:58
2
$\begingroup$

To complement Fadi's answer, the following are other useful papers on NL to SQL methods. The major difference of these methods is that they support queries that should be answered using more than one table (joining different tables), however the Salesforce paper (and their dataset) is focused on queries on one table at a time.

Both of these papers use the GeoQuery dataset avaialbe here.

Improve this answer
$\endgroup$
1
$\begingroup$

There are lots of works on text-to-SQL task.

I strongly suggest you to check WikiSQL and Spider datasets. Studies start from seq2seq + attention mech. to BERT-based solutions. Also each study points out the importance of the input representaion where you can feed all the table schema or just a column name. It's a pretty deep topic and as @PyRsquared said there is no simple answer :)

Improve this answer
$\endgroup$
1
$\begingroup$

Just to complement this thread with the latest AI updates on this matter: Open AI released recently Codex.

Codex is an AI trained to convert and analyze many coding languages (main focus is python).

It's also GPT-3 based.

Improve this answer
$\endgroup$
1
  • $\begingroup$ codex is terrible at this, the more you get deeper the more you understand how far it is from really work. it makes many syntax errors with GROUPS and subqueries $\endgroup$
    – hkrlys
    May 10 at 17:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.