I am looking for a model with which I can predict the probability of a current word given its n predecessors (or successors) in a sentence.
Please note: I do not want to generate text nor do I want to predict the next word, but rather I want to know if a given already existing word/sentence makes sense or not.
For this I am looking for a solution that solves the following two problems:
First: For example: Given the sentences "I build a house" and "I build a soup".
I want to have a model that tells me
P( "house" | "I build a") and
P( "soup" | "I build a")
So in this illustrative example I would like to get something like 30% for "house" and say 0.1% for "soup".
Second: This is related to the first requirement but now I want to ask what is the probability of any word in a sentence given the other words.
For example given again the sentence "I build a house" what is
P("build" | "I", "a house"). In this case I would expect the model to tell me that this word is reasonable within this context. While
P("build" | "I", "a soup") should be evaluated as unreasonable.
Which model would be recommendable for solving this problem? Ideally a pretrained one that is available for download.