Deep Learning book - PCA - showing the covariance of z is diagonal

Question

In the Deep Learning book at Page 146, the authors are showing that the covariance of the representation $z = W^T x$ is diagonal (Eq. 5.92 through Eq. 5.95).

The terms are conveniently arranged: $$ Var[z] = \frac{1}{m-1} Z^T Z = \frac{1}{m-1} W^T X^T X W = \ldots = \frac{1}{m-1} \Sigma^2 $$

However, I am stuck on: $$ Var[z] = \frac{1}{m-1} Z^T Z = \frac{1}{m-1} {(W^T X)}^T W^T X = \frac{1}{m-1} X^T W W^T X $$

Answer 1

So I’m not sure if I understand the question, but the thing that originally threw me off reading the book was the construction of Z.

$\mathbf z = W^T\mathbf x$ so I mistakenly assumed Z = WX, when in fact it should be Z = XW. In other words the columns of Z are the projections of the rows of X through W. Let me know if it's something else that's not clear.