How to create domain rules from raw unstructured text using NLP and deep learning techniques ? For example for the below text on symptoms of Dengue, all three look pretty similar but if you want to make sure a person is having Dengue you want to exact definite common minimum rules out of these raw text , so as to confirm a person is having Dengue. Can someone give reference to some research or blogs where similar problem has been solved ?

1) Symptoms of dengue fever include severe joint and muscle pain, swollen lymph nodes, headache, fever, exhaustion, and rash. 

2) High fever and at least two of the following:
Severe headache
Severe eye pain (behind eyes)
Joint pain
Muscle and/or bone pain
Mild bleeding manifestation (e.g., nose or gum bleed, petechiae, or easy bruising)
Low white cell count

3) Aching muscles and joints
Body rash that can disappear and then reappear
high fever
intense headache
pain behind the eyes
vomiting and feeling nauseous

For the above three extracts commons rule would look like

1) Fever
2) Joint and muscle pain
3) Headache
4) Rash
  • $\begingroup$ How could you define a labeled data for this type of problems? "exact definite common minimum rules out of this raw text, so as to confirm a person is having Dengue."? what an objective metric you could define? $\endgroup$ – Fadi Bakoura May 10 '18 at 18:39

I think your question can be solved using Case Based Reasoning .

Basic principle how it works is, you need to train the model using whole lot of different cases which you have. Based on the symptoms which you give the outcome is predicted(which disease).

Process Flow Please refer to the links below for deep diving into the topic, appended couple of links with respect to health care industry:

  1. Link -1
  2. Link -2
  3. Link -3
  4. Link -4
  5. Link -5
  6. Link -6
  7. Link -7

Do let me know if you need any additional information.

  • $\begingroup$ If you got what you were looking for, you can accept the answer. Let me know If you need any more explanation. $\endgroup$ – Toros91 Mar 21 '18 at 1:23

You can look into topic models like LDA to discover the most common topics. Preprocessing, like removing stop words, stemming and using n-grams and then applying LDA usually produces better results.

You can also use embedding together with neural nets to discover important words. Googles MLCC has a nice example.


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.