Minimizing error in cosine similarity

Question

Presume I have a vector space, and I am attempting to compress it into a latent vector space, while minimizing error in cosine similarity between entries. Suppose that I know the actual cosine similarity scores between the latent space. i.e.

Let $X \subset \mathbb{R}^n$ be my set of vectors, and $r: X \times X \to [-1, 1]$ be a function. I am trying to find a function $f: X \to \mathbb{R}^m$ such that the error $$\sum_{(x, y) \in X \times X} |sim(f(x), f(y)) - r(x, y)|$$ is minimized.

What is a good way of finding $f$ given $X$, $r$ and $m$? I am comfortable using gradient descent or neural networks.

Answer 1

To find the necessary function using error back propagation, you only need to design the architecture of the neural network, which will represent the desired function (for example, several layers, activation functions, etc.).

I do not see any problems when using the function you specified as a loss function. Because cosine similarity in this case is the product vector embeddings, then the gradient is calculated obviously.

I recommend that you study how face recognition models are trained (for example, Facenet). Very similar approaches are used there.

I'm pretty sure you're aware of the use of Kullback–Leibler divergence to solve this problem. I just wanted to draw attention to the possible use of SGD for its optimization, which may give a hint in your case.