I'm working on a project where I use FAISS to retrieve n neighbouring vectors based on a query vector. The data in question is textual and is being embedded by using a machine learning model to create a vector before it goes into FAISS.
These neighbors each have a category assigned to them, and also have a similarity score to the query, like the following:
Query: Berlin is the capital of Germany
=====
Neighbours output:
5 Neighbour ids: [57, 163, 177, 124, 91]
Text | Category | Similarity
Berlin is a great city to live in | Capital cities | 0.897843
Capital letters are often used to indicate nouns in German | Grammar | 0.803834
Over 3 million people live in Berlin | Capital cities | 0.79434
Germany is a country in Central Europe | Countries | 0.763232
Germany has many big cities | Countries | 0.7304545
Now, the thing I want to achieve is getting a single category for the query based on the categories of the neighbours, a kind of vector based recommender/suggestion system. What I tried already is just doing simple and weighted (based on similarity) majority voting.
Using simple majority voting, in the above example I would just get "Capital cities" or "Country" category as they both occur 2 out of 5 times. Using weighted voting I would arrive at "Capital cities" as they have higher similarity overall.
Both the approaches seem to work, however I am looking for a slightly more sophisticated approach to combine different signals. I read about the concept of data fusion in machine learning, but I don't quite know yet how to best apply it here to arrive at one category based on the neighbours.
Any ideas are appreciated!