4
$\begingroup$

I've been asked to report the number of parameters to be learned in a PCA model. This answer implies that parameters do exist in PCA, but does not explain. Software packages often report the number of parameters, but do not document what those parameters are.

What are the parameters in a PCA model? Subsidiary question: how many parameters are there?

To be clear - I am not asking about model hyperparameters.

$\endgroup$
1
  • $\begingroup$ What do you mean by a PCA model? $\endgroup$ – Dave Nov 14 '20 at 14:17
3
$\begingroup$

From an algebra point of view PCA is a base change. You can write the transformation as:

$$ T = XW $$

Where $X$ is an nxp matrix (n instance, p features) and $W$ is a pxp 'weight' matrix (whose columns are eigenvectors of $X^TX$ - translating one base to another). Hence the general PCA has $p^2$ coefficients.

You can also reduce the space to m dimensions and use :

$$ T = XW_m $$

Where $W_m$ is a weight matrix constituted of m first columns of $W$. $W_m$ is a pxm weight matrix and has pxm coefficients (or easier to understand : mxp coefficients, that is number of dimension kept and parameters needed to express the new base vector in terms of the initial base). In the case of a projection onto 2 dimensions you will have 2p coefficients.

$\endgroup$
1
  • $\begingroup$ So a PCA of rank m, over p-dimensional data has m*p parameters, and these parameters are the coefficients of the linear combination that represents the projection of our initial data into an m-dimensional space? $\endgroup$ – Chris Keefe Nov 19 '20 at 20:48

Not the answer you're looking for? Browse other questions tagged or ask your own question.