Context: NER (named entity recognition) and RE (relation extraction) from sentences obtained from radiology reports (medical text).

There is a BERT-based model from HuggingFace I would like to use for this: https://huggingface.co/microsoft/BiomedVLP-CXR-BERT-specialized

I already have experience using this model for simple tasks, such as comparing vector representations of sentences and classifying sentences. I've already written scripts for fine-tuning and evaluating this model on said tasks.

Now I want to do NER and RE using this model. I plan to use the following dataset for that purpose: https://physionet.org/content/radgraph/1.0.0/

Here is a figure from the paper:

enter image description here

Implementation-wise, I foresee the following challenges:

  • I need to obtain the token representations from the last layer of CXR-BERT.
  • I need to add a layer for NER on top of those tokens.
  • I need to represent entities.
  • I need to add a layer for RE given the representations of a pair of entities.
  • I need to convert the raw dataset into labels for NER and RE somehow, making sure that the labels are consistent with the tokenizer of CXR-BERT.
  • I need to define some appropriate loss function(s) to train the model. I've read that CRF (conditional random fields) are useful in this case, but I have zero experience using them.

I need help figuring out a way of implementing (coding) all of the points above. Ideally I'd like to make reasonable design decisions based on what usually works well for people doing NER/RE (I don't have much experience with those tasks, this is my first time).

Any suggestions and insights will be much appreciated.

Thanks a lot in advance.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.