I have embeddings of text created with a BERT model. A group of these embeddings should be used to find similar embeddings corresponding to this group. I know that you can use average or max (or concatenation) to combine multiple embeddings, which the can be used to for example calculate a cosine-similarity. But I wonder if there are other ways to do so, to even improve the result?
My Idea is to create an "artificial" embedding with weights to highlight important features. For that I'm calculating the mean for each dimension as well, but in a next step I am weighting the dimensions by their corresponding standard deviation. To be able to compare this new created artificial embedding, I normalize it to be more similar to the others.
Another idea is to reduce dimensions of the original embeddings to find the most important features (e.g. with PCA) and then calculate for example the mean. Nevertheless it will be difficult to compare the new embedding, as it then has a different size and the cosine-similarity-score is not possible to compute anymore.
What is a good approach to address this problem? Are there any state of the art ways or best practices to do it?
