I have search at lot, was not able to find a solution for my problem... I am training a NER model, that should detect two types of words: Instructions and Conditions. This is not the standard use-case of NER, as it does not search for specific types of words (e.g. Google == Corporation), but is rather much more depended on the sentence structure.

For example: If the car crashes, the airbag should go off.

  • 'crashes' should be labeld: "condition"
  • 'go' should be labeld: "instruction"

When training the model, I want to provide for each sentence not only my annotations, but also the dependency tree of the sentence calculated by the 'en_core_web_sm' model. I want my model to not only train based on the given words but also train based on the sentence structure.

My training data currently looks like this, but I want to expand it by also adding the dependency tree of each sentence generated using the 'en_core_web_sm' model:

train_data = 
    ("If the car crashes, the airbag should activate", [(11, 17, 'CON'), (38, 46, 'INS')]),

This is my current training loop, using the update function from spaCy, but I am open on trying a different tool:

import random 
import datetime as dt
from spacy.util import minibatch, compounding
from spacy.util import decaying

dropout = decaying(0.6, 0.2, 1e-4)

nlp = create_blank_nlp(TRAIN_DATA)
optimizer = nlp.begin_training()
for i in range(80):
    losses = {}
    batches = minibatch(TRAIN_DATA, size=compounding(4.0, 32.0, 1.001))
    for batch in batches:
        texts, annotations = zip(*batch)
            texts,               # batch of texts
            annotations,         # batch of annotations
            drop=next(dropout),  # dropout
    print(f"Losses at iteration {i} - {dt.datetime.now()} {losses}")

I am curious if and how this is might be possible. It feels like a waste to not use the pretrained model (mind you, the pretrained NER model from spaCy probably will not help me, only the dependency part).

Open to any advice, thank you.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.