2
$\begingroup$

I am looking for a python package that calculates how well one sentence of a natural text follows the next. One could simply count how many identical words are in the next sentence but the better method would be to compare word similarities using something like word vectors (=semantically similar words instead of exact matches or synonyms).

Coherent:

Tom loves reading books.
He prefers reading books at library.
So he always goes to library.

Non-coherent:

Tom loves reading books.
He missed his lunch today.
So he always goes to library.

I guess there must be several well written packages for such an automatic assessment (or similar methods) but I just can't find it. Any ideas?

$\endgroup$
1
$\begingroup$

This task looks similar to what is called text segmentation, in particular topic segmentation. I don't know any python package to do it but apparently Google gives a good few results for "semantic text segmentation python" (I'm not sure that this is the best phrase, you might want to try variations).

Note: this is still an active NLP research topic as far as I know. I don't know how fast reliable python packages are written and maybe there are some for this, but I wouldn't be too surprised if there were only research prototypes available at this stage.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.