- Suppose we have a data-set with 4 features.
- Suppose we calculate the PCA for this dataset and we plot the
scree-plot:
What we can learn from the features? Can we say that they are not linearly correlated? Can we say something else?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
In short: what you can learn is that you data variability is distributed on multiple of the orthogonal dimensions and PCA can not reduce it further.
Longer: In an extreme scenario, imagine you have four features/dimensions which have variability distributed across all dimension. It is hard to think of a four dimension object but think you have a perfect cube (three dimension) which each edge get various multiple color (the fourth dimension). Here is a dedicated post on understanding geometrically four dimension object
Since all these dimensions are orthogonal (in this example the color is not but just imagine it meets the statistical assumption of the PCA) the PCA can not do much. (In fact PCA of a cube, is a cube and nothing less).
What you can do : explore some non-linear dimension reduction - Principle Components from PCA are orthogonal - perhaps if you choose a non-linear method, you find subset of dimensions which can carry the variability - but you won't get anything less than four - but more PCs. Still I doubt you can reduce that it.
$\begingroup$
$\endgroup$
PCA provides the eigenvalue corresponding to each feature. This value is used to decide the number of features to retain after dimensionality reduction.
The features/pc are sorted in descending order based on their eigenvalues and the top N features are preserved after the dimensionality reduction.
The plot above shows bar's with eigenvalue's of each principal component represented as pc after performing PCA on the dataset.
