I am new to NLP and I just finished reading the paper "Attention is all you need". I'm struggling to understand the interpretability of the multi-headed attention, and specifically how these visualizations were produced:
I understand that the output of the self-attention sub-layer (for a single head) is a vector of size d_v that is a weighted sum of all the value vectors. Than how do they use this vector to calculate the strengths of the relations between the positions?
Any help and insight would be appreciated, thanks a lot!