As we see here, to build an NER model we can pass text features (parts of the word, pos tag, structure of the word etc.) to Sklearn-CRF. I was wondering how does this package convert the text features to numerical features? What strategy do they use - any specific type of embedding?


CRF models are not DL models, they use text features in the traditional way as categorical data (equivalent to one-hot-encoding). However they are different from other supervised ML methods because they exploit the sequential information of the text.

Typically the text sequence is represented word by word, and each word can have several additional "features" usually represented as columns. For example:

The     DET   the
dog     NOUN  dog
chases  VERB  chase
a       DET   a
cat     NOUN  cat

The real features used by the CRF are computed based on the rules defined before training the model. They can involve the position of the current word or any other position relative to the current one. The features are usually simple binary values, for examples "doc[n-1][2] == 'DET'" would represent the condition "the previous word is a determiner": "previous word" is indicated as n-1 in the sequence, column 2 indicates the POS column. The exact syntax to specify the rules may differ with different CRF implementations but the principle is the same.

  • $\begingroup$ Thanks for the reply. But as I see sklearn-crfsuit accepts the following type of features also: structure of the word - where a letter in the word will be represented by 'x', digits by '0' and special characters by '.' - so, the word 'w3.com' will be 'x0.xxx' How does the model architecture interpret these type o textual features? $\endgroup$ May 18 '21 at 9:51
  • $\begingroup$ @SaikatBhattacharya Certainly these different representations would be treated as additional columns, allowing a different kind of matching on the word. For example in my example you could have an additional column at index 3 and this column could be tested with a feature, for instance "doc[n][3] == 'x0.xxx'". Technically there is no limit to the number of features columns which can be considered. $\endgroup$
    – Erwan
    May 18 '21 at 22:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.