Bella Pharma Rosuvas 5 Enalapril 10 Domperidone 10 Ned's 24 by 7 PCM 650 Teneligliptin 5 ...
(1,2,shop), (3,4,med), (5,6,med), (7,8,med), (9,12,shop), (13,14,med), ... i.e. words 1 and 2 denote a shop name, 3 and 4 denote a medicine name, etc.
- recognize shop names: some shop names maybe non-English dictionary words (Romanized spelling of local language words)
- separate medicine names: there are absolutely no punctuation marks like
,and the pattern is like SmmmmSmmSmSmmmmmmmmSmm... (S is shop, m is full name of one medicine e.g.
Text is non-grammatical (just transcript of dictation between 2 people verifying inventory)
spaCy en_core_web_lg and other pre-build NER don't work (probably because both shop names and medicine names appear like proper nouns)
Also, there is no exhaustive list of medicine names: same chemical is sold under different names by different brands, sometimes weight/power specification may not be given.
Worth noting that chemical names have either suffix like *azole, *nate, *ide, *ril, etc. or weights like 650mg, 10mcg, etc. often (but not always) associated with them.
I am amazed how humans (even non-Pharmacists) can label most of the data correctly, how can I label this data almost as well as humans using ML/DL libraries?
I work as junior SDE, I can code just fine (e.g. convert BIO tag to JSON style spacy input) but have never used ML libraries before. Kindly include sample code in answer or link to same.