How to mathematically explain the translational and rotational invariance of PCA

Question

There is a homework question for a course I am self studying (not a student) that is:

let our $n \times d$-dimensional data vectors be denoted by $x_1,\ldots,x_n$ and let $R$ be a $d \times d$ rotation matrix. For simplicity, you may assume that the $x_t$'s have been centered at $0$. Let $$x_t' = Rx_t + v$$ where $v$ is some fixed translation. Forming a second dataset. Now, for any $K$ we pick, let us use PCA on each of the two data sets to obtain $K$-dimensional projections $y_1,\ldots ,y_n$ and $y_1',\ldots,y_n'$, respectively.

Write down a relationship between the two PCA projection matrices $W$ and $W'$ in terms of the rotation matrix $R$ and the translation vector $v$. Explain mathematically how you arrived at this answer.

My answer is basically that while for the untransformed dataset we have that $W$ is the top $k$ eigenvectors of $S[\mu] (S[\mu]$ is the covariance matrix of the untransformed dataset centered at $\mu (\mu$ is $0$ for untransformed dataset)) that $W'$ will be the top $k$ eigenvectors of the matrix $(R^T) \times S[R^T(\mu-v)] R$. ($R^T$ is $R$ transposed). I did this by applying the transformation in the definition of the covariance matrix $S'$ to find a relation between $S'$ and $S$.

The goal of the problem is to show the rotational and translational invariance of PCA. Can anyone give an explanation for this?

Answer 1

Let $e$ be the all-one column vector of length $n$.

Let $X\in \mathbb{R}^{n\times d}$ be the original matrix.

and $X'$ be the transformed matrix, that is we have $X'=XR^T+ev^T$.

Let's first compute the transformed mean assuming the mean of $X$ is $0$.

$$\frac{e^TX'}{n}=\frac{e^TXR'}{n}+\frac{e^Tev^T}{n}=v^T$$

To compute the singular vectors of the original matrix, theoretically, we can compute the eigenvalues of $$X^TX=UDU^T$$

And we can extract $W$ from $U$.

To compute the PCA for the transformed data

We have $$(X'-ev^T)^T(X'-ev^T)=(XR^T)^T(XR^T)=R^TX^TXR^T=RUDU^TR^T$$

Now, we can extract $W'$ from $RU$.

That is $$W'=RW.$$