I've just come across chunking and I can't get my head around why is it necessary? I know that it is used for 'named entity recognition'. I have few questions:

  • Why and how is Chunking helpful?
  • Plus aren't POS tags enough?
  • Why are three tags necessary for the representation? Why not just use 'inside' tag and 'outside' tag?

1 Answer 1


BIO(L) tagging is important (but as you correctly noted, not necessary) part of a NER pipeline. Main idea behind such split is to facilitate learning in following manner.

Take English as an example, some words will (most likely) never end a Named Entity, like adjectives, so the model will never tag them as the L(ast) part of a named entity. The same applies to the L-tag.

What is crucial, is that many models, like Conditional Random Fields, learn not only tags themselves, but also the transition probability, so, if you will get some chunk of text that is tagged as B_ O_ L_, that sequence is incorrect, but when you learn the transitions as well, model will figure out, that if you get a strong Beginning and End, the inside part should also be a entity part.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.