My task is to connect 2-3 parts of the sentence into one whole using a preposition

  • the first part is some kind of action. Ex. "take pictures"
  • the second part is an object that can consist of only one noun or a noun with adjectives and additions dependent on it. Ex. "juicy cherry pie", "squirrel"
  • the third part is a place. Ex. "room", "London"

To solve this task I've already tried some options such as generation using GPT-2 (or other networks like LSTM) and using grammar rules, but both of them didn't work as well as I want.

Using GPT-2 needs a lot of resources and performance, which is very expensive on the server. Ordinary neural networks aren't good enough and often generate bad results. And it is also impossible to describe all cases using only grammar rules.

Are there any approaches how you can implement a solution to this problem without using transformers or grammatical rules, so that the model isn't too heavy and at the same time effective?

There may be several options for prepositions, the main thing is that they are used.

P.s. a good option was to use n-grams, but it is not clear how to work with it when the second part is long.


[Pour, juice, boiler room] -> Pour the juice into the boiler room

[Go, theater] -> Go to the theater

[Learn the history, ballet] -> Learn the history of ballet

  • $\begingroup$ This might be interesting to you ramsrigoutham.medium.com/… $\endgroup$ Commented Apr 5, 2021 at 19:03
  • $\begingroup$ Thanks, its really cool idea, but BERT is transformers which are very expensive in production $\endgroup$ Commented Apr 6, 2021 at 8:51
  • 1
    $\begingroup$ Will DistilBert work for you in such case? $\endgroup$ Commented Apr 6, 2021 at 17:37
  • $\begingroup$ I have not tried to use it, but I'll try $\endgroup$ Commented Apr 7, 2021 at 18:26

1 Answer 1


The traditional approach for this kind of problem would be an n-gram language model. The language model is trained on a large corpus, then it's reasonably simple to calculate the most likely missing tokens for any incomplete sentence. SRILM was one of the most common toolkits, but there are probably many other libraries.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.