N-gram language model for preposition predition

Question

I am trying to build N gram models to predict the missing prepositions of a text corpus.

I would want to have some guidance on if I'm understanding and doing things correctly.

So the N gram model is basically just a collection of posterior probabilities? Pr(this word | previous words)? Then how is this machine learning I wonder? Since we would get a deterministic set of probabilities based on the frequencies of the word combinations from the training set. There doesn't seem to be any parameters to learn except in interpolation (like the weights of each gram in their weighted sum).

As for the actual prediction of preposition, after getting a set of the posterior probabilities of all the words in the vocabulary, do I simply only compare the posterior probabilities of the few known prepositions and find the argmax as the prediction?

Appreciate any help, thanks!

Answer 1

Your understanding is mostly correct, but don't forget that you can not only take into account the previous tokens, you can also consider:

• p(this word | next words)
• p(this word | N previous word,N next words)
• etc.

Often a combination of these probabilities offers optimal results.

how is this machine learning I wonder?

Well, ML is very often a deterministic calculation, it doesn't have to be a complex approximation problem: decision trees, Naive Bayes, linear regression...

do I simply only compare the posterior probabilities of the few known prepositions and find the argmax as the prediction?

Yes, that would be the idea. Of course if you use multiple models as suggested above there must be some kind of combination additionally.