Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I'm using the FTS5 functionality in Sqlite3 to compute a search rank, which uses BM25. Maybe I'm just not seeing it, but I can't find any description of how to actually interpret BM25's output ranking.
When I test on real data, all the ranks are negative decimals like -2.934393 or -4.29809.
Does a large value mean a result is more relevant to the search, or is it some sort of inverse log, so the smaller value is more relevant? Does "-2" indicate a better result than "-4"?
Reference https://www.sqlite.org/fts5.html Section -5.1.1. The bm25() function [![The bm25 score computation formula][1]][1]
The "-1" term at the start of the formula is not found in most implementations of the BM25 algorithm. Without it, a better match is assigned a numerically higher BM25 score. Since the default sorting order is "ascending", this means that appending "ORDER BY bm25(fts)" to a query would cause results to be returned in order from worst to best. The "DESC" keyword would be required in order to return the best matches first. In order to avoid this pitfall, the FTS5 implementation of BM25 multiplies the result by -1 before returning it, ensuring that better matches are assigned numerically lower scores.
So, based on the above explanation, the ranking in your results should follow the order -2>-4. PS- Because, no order info pertaining to the query as well the data is available, the most feasible explanation seems the one above. [1]: https://i.stack.imgur.com/PK3OM.png
Please let me know if the answer was helpful.
@Cerin My interpretation of the snippet is that the BM25(using fts) assigns numerically higher value for a better match and then returns the results in ascending order (by default), thus returning the ranks in the order from worst to best. But, for getting the best matches first, we need to order by DESC. So, to include best to worst order or DESC, (-1) is multiplied (as indicated in the formula) to converts the positive integers ranks into negative integer ranks. When these ranks are ordered in ascending (which is by default), we would get the smallest rank (best rank) at the top, followed by the largest at the bottom (worst rank). This should be best illustrated by an example. Say BM25(using fts) gives ranks based on the formula and without any ascending or descending sorting 2,4,6 When sorting is done in ascending order(by default), we would get in order2,4,6 with 2 being the worst rank and 6 being the best rank, as a better match gets a higher score. But, to accommodate the order best to worst, we multiply by -1 and then apply the ascending order, so that we get -6,-4,-2
So in your case, you are correct as -4 is a better than -2, going by the same logic.
Hope, that I could clarify stuff.
