1
$\begingroup$

I'm using the FTS5 functionality in Sqlite3 to compute a search rank, which uses BM25. Maybe I'm just not seeing it, but I can't find any description of how to actually interpret BM25's output ranking.

When I test on real data, all the ranks are negative decimals like -2.934393 or -4.29809.

Does a large value mean a result is more relevant to the search, or is it some sort of inverse log, so the smaller value is more relevant? Does "-2" indicate a better result than "-4"?

$\endgroup$

2 Answers 2

1
$\begingroup$

Reference https://www.sqlite.org/fts5.html Section -5.1.1. The bm25() function [![The bm25 score computation formula][1]][1]

Excerpt from the section

The "-1" term at the start of the formula is not found in most implementations of the BM25 algorithm. Without it, a better match is assigned a numerically higher BM25 score. Since the default sorting order is "ascending", this means that appending "ORDER BY bm25(fts)" to a query would cause results to be returned in order from worst to best. The "DESC" keyword would be required in order to return the best matches first. In order to avoid this pitfall, the FTS5 implementation of BM25 multiplies the result by -1 before returning it, ensuring that better matches are assigned numerically lower scores.

So, based on the above explanation, the ranking in your results should follow the order -2>-4. PS- Because, no order info pertaining to the query as well the data is available, the most feasible explanation seems the one above. [1]: https://i.stack.imgur.com/PK3OM.png

Please let me know if the answer was helpful.

$\endgroup$
1
  • $\begingroup$ Are you sure about this? In my tests, results that have, say, 2 occurrences of a search term have a rank of around -4, whereas results with only 1 occurrence have a score of around -2. That would indicate that -4 is indicates a more relevant result than -2. That also follows from your snippet, where it sounds like they just negate the rank, so a large rank value becomes a large negative rank value. $\endgroup$
    – Cerin
    Jul 23, 2022 at 15:45
0
$\begingroup$

@Cerin My interpretation of the snippet is that the BM25(using fts) assigns numerically higher value for a better match and then returns the results in ascending order (by default), thus returning the ranks in the order from worst to best. But, for getting the best matches first, we need to order by DESC. So, to include best to worst order or DESC, (-1) is multiplied (as indicated in the formula) to converts the positive integers ranks into negative integer ranks. When these ranks are ordered in ascending (which is by default), we would get the smallest rank (best rank) at the top, followed by the largest at the bottom (worst rank). This should be best illustrated by an example. Say BM25(using fts) gives ranks based on the formula and without any ascending or descending sorting 2,4,6 When sorting is done in ascending order(by default), we would get in order2,4,6 with 2 being the worst rank and 6 being the best rank, as a better match gets a higher score. But, to accommodate the order best to worst, we multiply by -1 and then apply the ascending order, so that we get -6,-4,-2

So in your case, you are correct as -4 is a better than -2, going by the same logic.

Hope, that I could clarify stuff.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.