Creating Domain specific Question Answering Systems

Question

I want to create a domain specific QA system.

I am working on a project to create an QA system for a given text book (on a specific domain and answer all related questions)

One approach I am considering for this is to create an Ontology/Knowledge base and then use this for answer retrieval.

For this :

1. How can I extract data from documents and populate this to the KB ?

2. The question will be in natural language, how can I use this question to query from the knowledge base ?

3. Is this the best approach ? because domain knowledge is required here for creating ontologies. And there are n number of domains. So shall I need to create different ontologies for each domain ?

4. Can a single QA model can be used to cover different domains ? Is there any other Machine learning and Deep learning approaches can be used for this ?

Answer 1

This is broad question, but here is the general approach. There are several sub-problems involved in a typical QA system, and each may be solved using different approaches:

1. Question Classification - What type of question is the user asking? This can be posed as a classification problem, assuming you have labels.
2. Parsing the question - Multiple NLP techniques would be required here
3. Converting question to canonical form - Every question would be converted into a canonical or structured format (which can be used to query). This is usually the most difficult part of the process. This is usually referred to as Semantic Parsing. One approach is to get variants of phrases and then map it to a single canonical form using some similarity metric. You can start with this paper.

If you can get from a natural language text to a canonical form, you are essentially done. Storing data and retrieval are trivial aspects which could be done in numerous ways. Coming to the question of domains, it is always going to be a trade-off between accuracy for one domain vs a general QA system (which is way too difficult to crack). A single model for every domain will most likely give you better results at the cost of extra effort.

Answer 2

You should check the Allen AI competition on kaggle: https://www.kaggle.com/c/the-allen-ai-science-challenge

In short, the typical approach people took there was similar to what you're suggesting:

• build a knowledge base from domain specific articles and/or Wikipedia
• index these articles with lucene or other IR system
• for each question/answer pair retrieve most relevant articles and use them as features
• build a classifier using these features for classifying if the answer is correct or incorrect