There are linear layers to modify the Query, key and value matrices and one linear layer after the multi head attention as they mention also from here:
Are these linear layers simply dense or fully connected layers? Let's consider the weight matrix Wi Q. Does this represent a dense layer with "Q" nodes? As they are using matrices as input rather than 1D vectors, I am getting a little confused.