# What can we learn from PCA on non linear data?

Suppose we have dataset with 10 features which are not linear:

    import numpy as np
from sklearn.decomposition import PCA
import matplotlib
import matplotlib.pyplot as plt

v1 = np.random.rand(100)

print (type(v1))

v2 = 2**v1
v3 = 3**v1 + np.matmul(v1, v1)
v4 = 4**v1 + np.matmul(v2, v3)
v5 = 5**v1 + np.matmul(v1, v3)
v6 = 6**v1 + np.matmul(v1, v4)
v7 = 7**v1 + np.matmul(v2, v2)
v8 = 8**v1 + np.matmul(v4, v5)
v9 = 9**v1
v10 = 10**v1

v = [v1,v2, v3, v4,v5, v6,v7, v8, v9,v10]

pca = PCA()
pca.fit(v)
pca.explained_variance_ratio_

PC_values = np.arange(pca.n_components_) + 1
plt.plot(PC_values, pca.explained_variance_ratio_, 'ro-', linewidth=2)
plt.title('Scree Plot')
plt.xlabel('Principal Component')
plt.ylabel('Proportion of Variance Explained')
plt.show() • I know that PCA is used to find the linear correlation. But what can we learn from that example?
• Can we use the PCA results and use only the first component of the PCA to train-predict our model?
• Does the result shown here is valid (correct for further processing)?

Not exactly an answer but just to say that, PCA is not a great way to approach non-linear relationships in your features. Imagine, trying to fit a linear regression to non linearly distributed data, like in the example below (only difference is PCA tries to minimise sum of squared distances conversely to residual sum of squares): Nonetheless, there is the chance PCA will be able to help measuring some of the data variance however keep in mind that even if no correlations exist in your dataset PCA will return PCs ranked in order of variance in each of your features.

I would suggest using kernel PCA, especially if you know there are non-linear relationships in your features.

There is another technique that works with non linear relations. Is called T-distributed Stochastic Neighbor Embedding - tSNE(https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html)

The author (Laurens van de Maaten) wrote a blog some time ago that I found very useful https://lvdmaaten.github.io/tsne/

t-SNE is a tool to visualize high-dimensional data. It converts similarities between data points to joint probabilities and tries to minimize the Kullback-Leibler divergence between the joint probabilities of the low-dimensional embedding and the high-dimensional data. t-SNE has a cost function that is not convex, i.e. with different initializations we can get different results.