Sklearn PCA with zero components example

Question

I'm simply trying to repeat a benchmark from the sklearn's docs.

The unclear part is: n_components = np.arange(0, n_features, 5). They are applying a PCA transform with 0 components!

Can somebody, please, explain, what's the mathematical meaning of this transform?

Answer 1

Think of it this way: a PCA "transform" with $k$ components essentially approximates your $n$-dimensional data points by projecting them onto a $k$-dimensional linear subspace, trying not to loose too much data variance along the way.

More precisely, what you are doing is representing your original points $y \in R^n$ as:

$$y \approx \mu + Vx$$

where $x \in R^k$ are the lower-dimensional "new coordinates", $\mu \in R^n$ is the mean of your data and $V\in R^{n \times k}$ is the matrix of principal component vectors.

The "new coordinates" $x$ tell you how many steps you need to make along the $k$ principal components in order to reach the best possible linear approximation to $y$ by starting your trip from $\mu$.

Now, if $k=0$ the model becomes:

$$ x \approx \mu. $$

In other words, you are modeling all of your data as a single, fixed center point. Of course, you do not need to store any "new coordinates" here (because you do not need to move away from the mean), hence it does not make much sense as a "transform", but it is, none the less, a proper probabilistic model (a maximum-likelihood fit for a Gaussian distribution of errors, to be precise).

In particular, you can speak about the log-likelihood of data under this model (which is, up to an affine transform, equal to the sum of squared errors here, but is not as trivial as you might think in the general case) and we can compare various models, choosing the one with the best likelihood. This is exactly what is done in the example from the docs you mention in the question.