2
$\begingroup$

I have search at lot, was not able to find a solution for my problem... I am training a NER model, that should detect two types of words: Instructions and Conditions. This is not the standard use-case of NER, as it does not search for specific types of words (e.g. Google == Corporation), but is rather much more depended on the sentence structure.

For example: If the car crashes, the airbag should go off.

  • 'crashes' should be labeld: "condition"
  • 'go' should be labeld: "instruction"

When training the model, I want to provide for each sentence not only my annotations, but also the dependency tree of the sentence calculated by the 'en_core_web_sm' model. I want my model to not only train based on the given words but also train based on the sentence structure.

My training data currently looks like this, but I want to expand it by also adding the dependency tree of each sentence generated using the 'en_core_web_sm' model:

train_data = 
    ("If the car crashes, the airbag should activate", [(11, 17, 'CON'), (38, 46, 'INS')]),
    ...
]

This is my current training loop, using the update function from spaCy, but I am open on trying a different tool:

import random 
import datetime as dt
from spacy.util import minibatch, compounding
from spacy.util import decaying

dropout = decaying(0.6, 0.2, 1e-4)

nlp = create_blank_nlp(TRAIN_DATA)
optimizer = nlp.begin_training()
for i in range(80):
    losses = {}
    batches = minibatch(TRAIN_DATA, size=compounding(4.0, 32.0, 1.001))
    for batch in batches:
        texts, annotations = zip(*batch)
        nlp.update(
            texts,               # batch of texts
            annotations,         # batch of annotations
            drop=next(dropout),  # dropout
            losses=losses,
        )
    print(f"Losses at iteration {i} - {dt.datetime.now()} {losses}")

I am curious if and how this is might be possible. It feels like a waste to not use the pretrained model (mind you, the pretrained NER model from spaCy probably will not help me, only the dependency part).

Open to any advice, thank you.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.