# There could be a problem with the linear layer after the attention inside a transformer?

My question regards this image:

It seems that after the multi head attention there is a linear layer as they mention also from here:

the linearity is given by the weights W^{o}. my quesion is: for the decoder, doesn't this linear layer mess up with the masking of the attention? I mean, if each token from the attention layer should not depend from the next tokens then these weights seem to mess up the whole thing or they don't? Indeed these linear weights will learn the dependencies among all the tokens and during inference there could be a problem, maybe(?). Thanks for any clarification.

If we zoom into the scaled dot product attention blocks, which happen before the projection with $$W^O$$ we see this:
There, you can see how the masking of the current and future positions happens inside the scaled dot product attention, which happens before the multiplication by $$W^O$$. Therefore, the values learned for $$W^O$$ are trained only with the information from the previous tokens in the decoder.