I've been assigned a task to extract features/attributes from product description.

Levi Strauss slim fit jeans
Big shopping bag in pink and gold

I need to be able to extract out attributes such as "Jeans" and "slim fit" or "shopping bag" and "pink" and "gold". The product description listings are not just for clothes, they can basically be anything.

I am not sure how to approach this problem. I tried implementing a Named Entity Recognizer solution and also a POS implementations, The NER implementation fails to recognize any token and most of the tokens show up as NNP(Proper Nouns) in he POS solution, which doesn't help me out a lot. I need a way to be able to distinguish between the brand name and the features of the Product(like if it is a t-shirt, the color or design(round neck, v-neck) etc).

I did implement a KMean solution which did cluster like products together, but then again it is not the result I am looking for.

Just looking for someone to direct me in the correct direction.

  • $\begingroup$ Did you train a custom NER on your data? $\endgroup$ – Emre Mar 15 '16 at 16:18
  • $\begingroup$ No I haven't tried training a custom NER. But there might be issues with that approach, I do not have a large enough training data set to train the NER, and even if I did have a large enough training data set, i'm not sure how the model would react to terms that were not present in the training data set or are from a different domain. $\endgroup$ – elric Mar 16 '16 at 5:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.