0
$\begingroup$

This is related to information extraction. In real world data, documents are written in bullets/numbered items form. For example,

How to create a website:  

 - Get A DNS
 - Get a Hosting 
 - Deploy wordpress or some site ...

above is sample of a structured data. Take another example where content is semi structured,

While sandeep was going to home there was a road on the way he saw a 

 - Car
 - 2 wheeler
 - cart     

and he carefully crossed the road

If I have to find out the "steps to create a website" or "what is on the road" .. is there an established method using NLP? As some data is semi structured, so simple classification may not work in this case.
Maybe deep learning network or some pretrained model? I have seen google is able to return such results when we search something like "what are steps create a website".

$\endgroup$
0
$\begingroup$

If we want to search for What are the steps to create a website, all you need to do is the find which set of bulleted points is going to answer that. One thing that you can try is that you can vectorize your question using some pretrained model (w2v, GloVe etc) and also encode the descriptions before the bulleted list ( For eg : How to create a website). Once you have the query vector $q$ and the descriptions vectors $d1, d2, d3,..., dn$ , you can perform cosine similarity of the query vector with all the description vectors and choose the description with the maximum similarity. Following this, the corresponding bullet points will be the answer.

$\endgroup$
  • $\begingroup$ thanks for reply. There are few issues with this approach (we have used this already): 1. Accuracy is low. 2. For semistructured data it is hard to find the title of these bullets. 3. When the answer can be in bullet or a non bulleted paragraph then it is hard to detect whether all bullets are the answer or just 1 bullet or the paragraph $\endgroup$ – Sandeep Bhutani Sep 5 '19 at 9:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.