Suppose we have dataset with 10 features which are not linear:

import numpy as np
from sklearn.decomposition import PCA
import matplotlib
import matplotlib.pyplot as plt

v1 = np.random.rand(100)

print (type(v1))

v2 = 2**v1
v3 = 3**v1 + np.matmul(v1, v1)
v4 = 4**v1 + np.matmul(v2, v3)
v5 = 5**v1 + np.matmul(v1, v3)
v6 = 6**v1 + np.matmul(v1, v4)
v7 = 7**v1 + np.matmul(v2, v2)
v8 = 8**v1 + np.matmul(v4, v5)
v9 = 9**v1
v10 = 10**v1

v = [v1,v2, v3, v4,v5, v6,v7, v8, v9,v10]

pca = PCA()

PC_values = np.arange(pca.n_components_) + 1
plt.plot(PC_values, pca.explained_variance_ratio_, 'ro-', linewidth=2)
plt.title('Scree Plot')
plt.xlabel('Principal Component')
plt.ylabel('Proportion of Variance Explained')

enter image description here

  • I know that PCA used to find linear correlation. But what can we learn from that example ?
  • Can we use the PCA results and use only the first component of the PCA to train-predict our model ?
  • Does the result show here are valid (correct for further processing) ?

Not exactly an answer but just to say that, PCA is not a great way to approach non-linear relationships in your features. Imagine, trying to fit a linear regression to non linearly distributed data, like in the example below (only difference is PCA tries to minimise sum of squared distances conversely to residual sum of squares):

enter image description here

Nonetheless, there is the chance PCA will be able to help measuring some of the data variance however keep in mind that even if no correlations exist in your dataset PCA will return PCs ranked in order of variance in each of your features.

I would suggest using kernel PCA, especially if you know there are non-linear relationships in your features.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.