Jurafsky's book says we need to add context to left and right of a sentence:

"Note that for these larger ngrams, we’ll need to assume extra context for the contexts to the left and right of the sentence end."

Does this mean,

for example, if we've a corpus of three sentences: "John read Moby Dick", "Mary read a different book", and "She read a book by Cher"; and after training our tri-gram model on this corpus of three sentences, we need to evaluate the probability of a sentence "John read a book", i.e. to find $P(John\; read\; a\; book)$ as below,

$P(John\; read\; a\; book)$
$=P(<s><s>John\; read\; a\; book<\backslash s><\backslash s>)$
$=P(John|<s><s>) P(read|<s>John) \; P(a|John\; read) P(book|read\; a) P(<\backslash s>|a\; book)\; P(<\backslash s>|book<\backslash s>)$
$=\frac{1}{3}\frac{1}{1}\frac{1}{1}\frac{1}{2}\frac{0}{1}\frac{1}{1}$ (without smoothing)

It would be great, if you let me know if the above understanding is correct?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.