1
$\begingroup$

I need to compare the price of the local drug-list of my country with prices of drug-lists of other countries (CSV format), I have a manually-matched table whic is my target and I use python via Jupyter on windows.

In the following, there are two records of two different drug-lists

one drug in list 'A'

Name:   40MG ABSEAMED*

Form:   injectable solution

Dosage: 0.9%

matched drug in list 'B'

Name:   ABSEAMED SOLUTION FOR INJECTION 40MG/0.4ML

Form:   SOLUTION FOR INJECTION

Dosage: 25mg/ml

My simple process is to clean data from undesired symbols and spaces, then to count occurances of features.

Problems:

  • using simple distribution function (intersected words in both) / (words of list 'A') the name of drug (ABSEAMED) is considered as priority as other features ('FOR', 'SOLUTION', ...)
  • By removing the Form words from Name words, some names becomes null
  • after cleaning data, I get noisy features (25mg/ml ===> 25 mg ml) and this dropped the accuracy of matching from 56% to 24%

What are the basic recommended techniques/tools (Automates, POS_tagger, ...) of Natural Language Processing that may give a high-accuracy results ?

I thought about creating my own POS tag to match tags before contents

(e.x. the drug name '0.225% W/V SODIUM CHLORIDE (1/4 NORMAL SALINE)

can be tagged as \dosage \symbol \name \quote*

but I'm not sure enough that it can improve my result

Any help is appreciated.

$\endgroup$

1 Answer 1

0
$\begingroup$

One of the workable approaches:

If you have sufficient data, and, you need to so this often, you can train your own custom NER model. Write a pipeline to match drugs and compare price and deliver a small app that solves the problem once and for all!

Custom Named Entity Recognition models helps you identify a named entity from a given chunk of text. There are plenty of models available that provide out of the box support of healthcare related named entities, like, drug! Once you have extracted named entities like drug, dosage etc. (btw, dosage can also be solved by writing a regex), you can compare the cosine similarities to find the right match. You can try different word embeddings to have more accurate match.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.