How effective is it to use BM25 to rank words, to be more specific i have a dictionary of words and i want to rank only words in a document that are also in my dictionary. I want to rank all words in my dictionary for each document and then add the BM25 value of each word for the specific document.
Lets say i have a document and dictionary like this:
myDictionary=['bad', 'dangerous','hide', 'following]
corpus=[
['human', 'intelligence', 'computer','bad', 'dangerous'],
['survey', 'user', 'human', 'system', 'time', 'hide', 'following],
]
Now iam going to run the below BM25 formula in a loop for all words in my dictionary and then sum the results of each word to get a bm25 value for each document.