I have unstructured text like this

From: Gujarat, IN 
To: Kerala, IN
Milk and egg, 100kg 
Please handle with care. 

(Tooth Brush and Paste)120KG in total
P/U: London, GB
Dest: QC, Canada
Keep the product away from heat
F:111-222 333

Like these, I have lakhs of samples with different products

From the above 2 samples, I want to extract the product name

From sample 1 Milk,egg from sample 2 Tooth Brush,Paste.

Things I have done so far

Based on my research it seems to be a NER problem

I worked on Custom NER using Spacy to extract the product from the samples but it is not efficient, I watched some videos related to NER and also explored similar content in stack overflow but it didn't work well for my problem.

How can I approach this problem further ?

  • $\begingroup$ What have you tried so far? Can you show us the code? $\endgroup$ – WBM Mar 19 at 12:59
  • $\begingroup$ code_where_I've_learnt_from @WBM $\endgroup$ – ConMan77 Mar 19 at 13:58

It's not a great dataset for NER because normally NER relies on trigger words close or inside the NE. For example "he went to X" would indicate that X is a location. Here there's no usable context and even the order of the lines can change.

What I would try here is to classify the lines, because apparently there's a kind of structure to each sample:

  • sender/recepient
  • email address
  • phone no
  • website
  • Instructions (e.g. "handle with care")

Once all the types of lines which are not a product are eliminated, the ones remaining are likely to describe the product.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.