# How to mathematically explain the translational and rotational invariance of PCA

There is a homework question for a course I am self studying (not a student) that is:

let our $$n \times d$$-dimensional data vectors be denoted by $$x_1,\ldots,x_n$$ and let $$R$$ be a $$d \times d$$ rotation matrix. For simplicity, you may assume that the $$x_t$$'s have been centered at $$0$$. Let $$x_t' = Rx_t + v$$ where $$v$$ is some fixed translation. Forming a second dataset. Now, for any $$K$$ we pick, let us use PCA on each of the two data sets to obtain $$K$$-dimensional projections $$y_1,\ldots ,y_n$$ and $$y_1',\ldots,y_n'$$, respectively.

Write down a relationship between the two PCA projection matrices $$W$$ and $$W'$$ in terms of the rotation matrix $$R$$ and the translation vector $$v$$. Explain mathematically how you arrived at this answer.

My answer is basically that while for the untransformed dataset we have that $$W$$ is the top $$k$$ eigenvectors of $$S[\mu] (S[\mu]$$ is the covariance matrix of the untransformed dataset centered at $$\mu (\mu$$ is $$0$$ for untransformed dataset)) that $$W'$$ will be the top $$k$$ eigenvectors of the matrix $$(R^T) \times S[R^T(\mu-v)] R$$. ($$R^T$$ is $$R$$ transposed). I did this by applying the transformation in the definition of the covariance matrix $$S'$$ to find a relation between $$S'$$ and $$S$$.

The goal of the problem is to show the rotational and translational invariance of PCA. Can anyone give an explanation for this?

• You can use LaTeX with \$ notation on this site – Martin Thoma Jan 11 '19 at 7:17
• Please note: PCA is NOT rotationally invariant. Only if you rotate all data, but it's pretty hard to find a method which is not rotationally invariant in that sense. Usually, people call an (image recognition) algorithm rotationally invariant, if you can rotate single images and not influence the output – Martin Thoma Jan 11 '19 at 7:20

## 1 Answer

Let $$e$$ be the all-one column vector of length $$n$$.

Let $$X\in \mathbb{R}^{n\times d}$$ be the original matrix.

and $$X'$$ be the transformed matrix, that is we have $$X'=XR^T+ev^T$$.

Let's first compute the transformed mean assuming the mean of $$X$$ is $$0$$.

$$\frac{e^TX'}{n}=\frac{e^TXR'}{n}+\frac{e^Tev^T}{n}=v^T$$

To compute the singular vectors of the original matrix, theoretically, we can compute the eigenvalues of $$X^TX=UDU^T$$

And we can extract $$W$$ from $$U$$.

To compute the PCA for the transformed data

We have $$(X'-ev^T)^T(X'-ev^T)=(XR^T)^T(XR^T)=R^TX^TXR^T=RUDU^TR^T$$

Now, we can extract $$W'$$ from $$RU$$.

That is $$W'=RW.$$