# How to mathematically explain the translational and rotational invariance of PCA

There is a homework question for a course I am self studying (not a student) that is:

let our $$n \times d$$-dimensional data vectors be denoted by $$x_1,\ldots,x_n$$ and let $$R$$ be a $$d \times d$$ rotation matrix. For simplicity, you may assume that the $$x_t$$'s have been centered at $$0$$. Let $$x_t' = Rx_t + v$$ where $$v$$ is some fixed translation. Forming a second dataset. Now, for any $$K$$ we pick, let us use PCA on each of the two data sets to obtain $$K$$-dimensional projections $$y_1,\ldots ,y_n$$ and $$y_1',\ldots,y_n'$$, respectively.

Write down a relationship between the two PCA projection matrices $$W$$ and $$W'$$ in terms of the rotation matrix $$R$$ and the translation vector $$v$$. Explain mathematically how you arrived at this answer.

My answer is basically that while for the untransformed dataset we have that $$W$$ is the top $$k$$ eigenvectors of $$S[\mu] (S[\mu]$$ is the covariance matrix of the untransformed dataset centered at $$\mu (\mu$$ is $$0$$ for untransformed dataset)) that $$W'$$ will be the top $$k$$ eigenvectors of the matrix $$(R^T) \times S[R^T(\mu-v)] R$$. ($$R^T$$ is $$R$$ transposed). I did this by applying the transformation in the definition of the covariance matrix $$S'$$ to find a relation between $$S'$$ and $$S$$.

The goal of the problem is to show the rotational and translational invariance of PCA. Can anyone give an explanation for this?

• You can use LaTeX with \$ notation on this site Jan 11, 2019 at 7:17
• Please note: PCA is NOT rotationally invariant. Only if you rotate all data, but it's pretty hard to find a method which is not rotationally invariant in that sense. Usually, people call an (image recognition) algorithm rotationally invariant, if you can rotate single images and not influence the output Jan 11, 2019 at 7:20

Let $$e$$ be the all-one column vector of length $$n$$.

Let $$X\in \mathbb{R}^{n\times d}$$ be the original matrix.

and $$X'$$ be the transformed matrix, that is we have $$X'=XR^T+ev^T$$.

Let's first compute the transformed mean assuming the mean of $$X$$ is $$0$$.

$$\frac{e^TX'}{n}=\frac{e^TXR'}{n}+\frac{e^Tev^T}{n}=v^T$$

To compute the singular vectors of the original matrix, theoretically, we can compute the eigenvalues of $$X^TX=UDU^T$$

And we can extract $$W$$ from $$U$$.

To compute the PCA for the transformed data

We have $$(X'-ev^T)^T(X'-ev^T)=(XR^T)^T(XR^T)=R^TX^TXR^T=RUDU^TR^T$$

Now, we can extract $$W'$$ from $$RU$$.

That is $$W'=RW.$$