# GloVe vector representation homomorphism question

In the paper GloVe: Global Vectors for Word Representation, there is this part (bottom of third page) I don't understand:

I understand what groups and homomorphisms are. What I don't understand is what requiring $F$ to be a homomorphism between $(\mathbb{R},+)$ and $(\mathbb{R}_{>0},\times)$ has to do with making $F$ symmetrical in $w$ and $\tilde{w}_k$.

Am I misunderstanding something? We want $F$ to be unchanged if we either interchange $w_i$ and $\tilde{w}_k$ OR interchange $w_j$ and $\tilde{w}_k$, right? Is this the only way to achieve the symmetry between $w$ and $\tilde{w}_k$?

If you're asking if the group homomorphism makes the the process symmetric then no it doesn't directly. However, they use the fact that they require a group homomorphism to show that $w_{i}^{T} \tilde{w}_k = log(P_{ik})=log(X_{ik}) - log(X_{i})$ This nearly gives us symmetry. Finally by adding $\tilde{b}_{k}$ into the equation you restore symmetry.

So in short $w_{i}^{T} \tilde{w}_k + b_{i} + \tilde{b}_{k} = log(X_{ik})$ is what ensures symmetry, and the group homomorphism is a tool to get there.

Update:

Some more details Essentially, what we want is the ability to peform a label switch. Group homomorphism helps with this process because it perseves a mapping between the $(R, +)$ and $(R, x)$.

$F((w_{i}^{T} - w_{j}^{T})w_{k}^{'})=F(w_{i}^{T}w_{k}^{'}+( - w_{j}^{T}w_{k}^{'})) = F(w_{i}^{T}w_{k}^{'}) \times F(-w_{j}^{T}w_{k}^{'} )= F(w_{i}^{T}) \times F(w_{j}^{T}w_{k}^{'})^{-1} = \frac{F(w_{i}^{T}w_{k}^{'})}{F(w_{j}^{T}w_{k}^{'})}$

The group homomorphism here allows for that to occur. Therefore we can see that by setting $F(w_{i}^{T}w_{k}^{i}) = \frac{X_{ik}}{X_{i}}$

Now finally we can say that $w_{i}^{T} {w}_k^{'} = log(P_{ik})=log(X_{ik}) - log(X_{i}).$

So as far as your comment, it is the most sensible chocie for their method and of which they buld the core mathematicals to GloVE. Changing it, I imagine wouldn't be a trivial thing. I imagine if you did, much of what is derived, including the loss function would change. But with that said, I imagine there are otherwise to achieve label switching.

• So this is a 'sensible choice' on their part, right? It's not strictly necessary for F to be that homomorphism for there to be symmetry in w and w~, right? Jan 25, 2018 at 16:31
• Depends what you mean by sensible. It's sensible because it gives them their property, but with that said, I would also state that it's a core property of their method. Jan 25, 2018 at 16:56