Let's say I made a PCA in which I reduced from 10 dimensions to 3. And it clusters the classes correctly, but how do I explain which dimensions are better to predict? It is obvious that the 3 dimensions are going to predict correctly and I can order those 3 to know which one is more useful to predict, but I don't know what are those 3 dimensions, they are just new dimensions, how could I explain which original features are the best?

I use scikit learn PCA btw. I'llbe gratefull if anyone can answer me.


1 Answer 1


Principal Component Analysis (PCA) is a technique used for dimensionality reduction. It transforms the original features into a new set of linear combinations called principal components. These components are uncorrelated and ordered in a way that the first few capture most of the variation present in the original features.

When reducing from 10 dimensions to 3 using PCA, the resulting 3 dimensions are not directly related to the original features. Instead, they are combinations of the original features.

To understand how the original features contribute to these principal components, you can examine the components_ attribute of the fitted PCA object in scikit-learn. This attribute provides an array where each row represents a principal component, and each column corresponds to the original features. The values in this array represent the weights of the original features in the principal components.

Here is an example:

from sklearn.decomposition import PCA

# Assume X is your data
pca = PCA(n_components=3)


The output might look something like this:

[[ 0.5  -0.1  0.2  -0.3  0.1  -0.2  0.4  -0.3  0.1  -0.4 ]
 [-0.3  0.4  -0.1  0.2  -0.3  0.1  -0.2  0.3  -0.1  0.2 ]
 [ 0.2  -0.1  0.4  -0.1  0.2  -0.1  0.3  -0.1  0.2  -0.3 ]]

In this example, the first principal component is 0.5 times the first original feature minus 0.1 times the second original feature plus 0.2 times the third original feature, and so on. You can see which original features contribute most to each principal component by looking at the magnitude of these weights.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.