1
$\begingroup$

I have multiple vector fields in one collection. My use-case is to find similar sentences in similar contexts. The sentences and contexts are encoded to float vectors. Therefore, I have one vector for the sentence and another vector for the context (surrounding text). I would like take both vectors in consideration to find similar sentences. Unfortunately, most approximate nearest neighbor (ann) search libraries only support to search for one field. I have tried to use PostgreSQL with the cube extension to filter by multiple vector similarities. Unfortunately, the number of vectors (100M) are too high for PostgreSQL.

Questions:

  1. Is there a possibility to combine multiple vector fields for approximate nearest neighbor search?
  2. Is it also possible to weight the relevance of each vector field for the search?
$\endgroup$

1 Answer 1

0
$\begingroup$

One alternative is to re-encode the sentences and context together into the same vector space. This can be done with doc2vec or StarSpace.

If the sentences and contexts are in the same vector space, any approximate nearest neighbor (ann) search libraries could work.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.