3
$\begingroup$

I have some text with different lengths, I want to split it into separate clauses but I also want to preserve the subject

For example;

# single subject
Original: "Coffee is very good, but wasn't hot enough"
split: ["Coffee is very good", "Coffee wasn't hot enough"]

Original: "Joe was the top performer of last year's dance competition, he is also a good singer"
split: ["Joe was the top performer of last year's dance competition", "Joe is a good singer"]

# multiple subjects
Original: "Delicious food service, but we struggled with the app."
split: ["Delicious food service", "We struggled with the app"]

I don't know how to achieve this, we can maybe split sentences based on punctuation and conjunctions (may not be accurate) but how do we preserve its subject.

Please let me know if you need more information.

$\endgroup$
1
  • $\begingroup$ I thought you were asking something else, and was about to ask what you used to get the output in your splits; especially putting Coffee in the first one is clever. But now I see that this is your desired output! So, as Erwan says in his answer, this is non-trivial. If you find something doing it out-of-the-box, do let us know! $\endgroup$ Commented Mar 8, 2022 at 9:05

2 Answers 2

3
$\begingroup$

This is not really sentence segmentation because the input is already a single sentence.

This is close to relation extraction but I don't know if it's applicable here. It's clear that there needs to be at least some minimal semantic analysis done in order to identify the subject. This could be done with dependency parsing, possible followed by semantic role labeling.

Note that things can get complicated if the sentence is in the passive form or in various other cases of paraphrase, e.g. "the top performer last year was Joe".

$\endgroup$
3
  • $\begingroup$ These things are quite new for me, can you recommend some resources or can you show a small example with code if possible. Thanks $\endgroup$ Commented Mar 7, 2022 at 17:22
  • 1
    $\begingroup$ Dependency parsing is quite standard so you'll find many resources, for instance this short tutorial. Semantic role labelling is less common and I'm not sure whether there are standard libraries. There are various academic resources though. Finally the task that you're trying to do is quite specific as far as I know, you will probably have to design your own method. If you want a general and accurate system, it's quite advanced NLP. $\endgroup$
    – Erwan
    Commented Mar 7, 2022 at 17:34
  • $\begingroup$ Thanks for sharing good resources @Erwan $\endgroup$ Commented Mar 8, 2022 at 9:30
2
$\begingroup$

After a lot of research, I figured out how to replace pronouns with their respective subject. It makes use of neuralcoref which is a pipeline extension for spaCy 2.1+ which annotates and resolves coreference clusters using a neural network.

However, it only works with spacy v2 and python3.7 I tested it on conda environment with following tools version

python==3.7
spacy==2.1.0
neuralcoref

The solution goes like this

import spacy
import neuralcoref

nlp = spacy.load('en_core_web_sm')
neuralcoref.add_to_pipe(nlp)

doc = nlp("Coffee is good but it wasn't hot enough!")
print(f'\n[REPLACED]:\n{doc._.coref_resolved}')

# output
Coffee is good but Coffee wasn't hot enough!

More sample outputs

[Enter your text]:
Joe was the top performer of last year's dance competition, he is also a good singer

[REPLACED]:
Joe was the top performer of last year's dance competition, Joe is also a good singer

[CONTINUE(Y/N)?]: y

[Enter your text]:
Paul was amazing and so was our waiter I loved the squash pizza and the dessert he recommended will definitely come back soon.

[REPLACED]:
Paul was amazing and so was our waiter I loved the squash pizza and the dessert Paul recommended will definitely come back soon.

I am still trying to figure out, a better way for splitting the sentence. Will update the answer once I figured this out.

$\endgroup$
1
  • $\begingroup$ Did you get better way of performing Symantic Sentence Segmentation? $\endgroup$ Commented Apr 19 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.