What are the approaches for extracting an injury and its description from a paragraph?

Question

Suppose I have a paragraph which explains the injuries and its descriptions. I want to extract the injuries and its corresponding descriptions from the text. How can I do that?

For example, the paragraph will be as follows:

In my opinion the neck pain is due to the soft tissue injury. The fracture on the hand will be resolved in 2 months. The pain in the shoulder and neck is due to the soft tissue injury. There is a stiffness and discomfort around the hip.

the expected output is :

{
"neck": ["soft tissue"],
"hand": ["fracture"],
"shoulder": [ "soft tissue"],
"hip": ["stiffness", "discomfort"]
}

Which NLP techniques can be used here?

We have two txt files for injuries and descriptions.

But how will we relate or match the description with its corresponding injury?

I tried the dependency parser but the problem is we have to write a number of patterns for each injury, we have more than 100 injuries and more than 100 descriptions. So if we are writing patterns for all the injuries there will be a large number of patterns and I think it will take too much time and power.

Are there any other ways to do this kind of extraction?

The paragraph doesn't have a common structure.

I'm using python and spacy for this.

Answer 1

Given the unstructured nature of your injury descriptions, I don't think this is doable by means of classical NLP techniques. I suggest you use a large language model (LLM), either OpenAi's GPT family or something like Llama or RedPajama. Give it a prompt with an example and it should give you the result.

This would be an example of a possible prompt using the example in your question:

Given the description of the state of a patient, extract the diagnosis of their injuries:

Description: In my opinion the neck pain is due to the soft tissue injury. The fracture on the hand will be resolved in 2 months. The pain in the shoulder and neck is due to the soft tissue injury. There is a stiffness and discomfort around the hip.

Injuries: {
  "neck": ["soft tissue"],
  "hand": ["fracture"],
  "shoulder": [ "soft tissue"],
  "hip": ["stiffness", "discomfort"]
}

Description: the butt pain is due to the coccyx bone. The bruise of the arm is due to the soft tissue injury.

Injuries: {

The model would complete the injuries JSON for you. You would then parse it. Given the lack of diversity in your example, you probably need to provide a couple more of examples and possibly with a wider variety of injuries. Designing an effective prompt (aka "prompt engineering") is part of using LLMs.

Note that you don't need to retrain the model, you can just use pre-trained models as-is, providing a sensible prompt that makes the LLM give you the desired outputs.

As for which model to use, there are dozens nowadays. Some are general domain, some are trained on medical data. The licence of some of them allows commercial use, and others only allow research uses. Some are very large, and others are smaller. You should research the currently available pre-trained models and choose the one that gives good results while meeting your operational constrains.