0
$\begingroup$

Say I have to predict the next word in a sentence, given the initial few words.

Suppose the prefix is "I went to _____". This prefix is common enough that it might appear 10 times in the training data with a few different variations:

I went to college: 5

I went to California: 3

I went to London: 2

In such a case, suppose my model predicts "college" as the right answer. It would still get only a 50% score if I used accuracy as a metric, the data naturally has multiple (correct) answers for the same input. How do I solve this?

$\endgroup$
0
$\begingroup$

Your accuracy should be this: Did the phrase existed before? It means either storing all combinations of 4-grams or searching the data on the fly.

But then the prediction algorithm that uses any combination from the previous data will be 100% accurate. I do not really understand the reason for such accuracy estimation.

$\endgroup$
0
$\begingroup$

Take this with a grain of salt (I'm super new to this stuff), and I'm assuming this is only a piece to the puzzle, but maybe someone with more expertise can build on this. I wonder, though, if utilizing a Levenshtein distance to quantify the accuracy of the string would be helpful. I got some pretty accurate correlations from a previous project, though I've admittedly never used it for NLP. If this is a terrible idea, I'd be interested to know why...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.