I would like to return a simple “true” or “false” for a given string which determines whether it is an incomplete sentence, like “which is why they usually”, or a complete entity, such as a chapter title or element in a list, for example “The boardwalk”.
I am open to using any machine learning architecture for this task and it’s possible I would provide context for the strings by showing surrounding lines for the algorithm to see if the sentence fragment fits in to some continuing text or not, but for a first draft I’ll probably just focus on the individual lines in isolation.
What would be the most standard way to approach this?
For example, maybe there’s already some pre-trained model that knows if something is a sentence fragment or not, which I could use.
Otherwise, if not, does it matter what kind of neural network I use, or is there a most standard one from PyTorch?
Is there a clever way to avoid needing to find or create training data? Can an unsupervised learning method naturally group lines of text into those which are similar and those which are different?
Or if I need to train it, are there algorithms which learn quickly, perhaps from just 100 examples?
Otherwise, I guess I will have to find or create a large source of data to train it on?
What would be the best way to approach this?
