0
$\begingroup$

I want to reduce the dimensions of my dataset to decrease the complexity without losing too much information.

But I am not sure how many dimensions I should reduce to, so that I don't lose too much information.

Do we have any direct attribute/function to use for such task?

Thanks in advance.

$\endgroup$
1
$\begingroup$

Being PCA the same as SVD, I'll explain the approach with the last one.

SVD decomposes a matrix $A$ in three others, so that $A = U * \Sigma * V^t$, where $\Sigma$ is diagonal consisting of $A$'s singular values in descending order.

The sum of all the singular values of $A$ is known as energy of matrix $A$. You want to retain most of the matrix's energy when reducing dimensions, so what you have to do is calculate how many top singular values you need to retain to preserve the most of the energy.

There are two ways of achieving this:

  1. 'Elbow method': plot the singular values and see from which on the energy isn't adding much to the total energy anymore.

  2. calculate how many of the top singular values you need to preserve most of the energy. Usually you want to preserve $\simeq 90\%$ of the energy. So you have to find $m / \frac{\sum_{i=0}^{m} sv_i}{\sum_{i=0}^{n} sv_i} \simeq0.9$.

I recommend you expand my answer with the links below. I especially recommend the mentioned book.

$\endgroup$
2
  • $\begingroup$ Thank you @89f3a1c for sharing your knowledge on that point. will check your suggestion. Though i have got one more solution for this problem(adding in answer), please have a look. $\endgroup$ – Santosh Kumar Nov 3 '19 at 8:42
  • $\begingroup$ Your answer makes sense to me, too. I just found an answer about PCA and proportion of variance which may help understand your approach, for anyone looking for further explanations on why the method could work. $\endgroup$ – 89f3a1c Nov 3 '19 at 12:59
0
$\begingroup$

I have used one inbuild function to highlight the variance of each principal component. Ultimately, the target is to cover more variance with minimal Principle component.

X is one dataset with 4 dimensions, initially, PCA will be applied on X with no dimension reduction, which means n_components = 4.

pca = PCA(n_components=4)
X_pca = pca.fit_transform(X) 

explained_variance_ratio_ is function which helps.

pca.explained_variance_ratio_

after running it. The variance ratio for each principal component came as below.

array([0.72962445, 0.22850762, 0.03668922, 0.00517871])

Now, it can be concluded that the first two PCs are covering 95% of the variance. so, 2 dimensions would be optimum choice to go ahead with.

pca_2 = PCA(n_components=2)
X_pca_2 = pca_2.fit_transform(X)

This further can used for learning. and If we want to draw a scatter graph for PC1 and PC2 for more clarity.

df_pca_2 = pd.DataFrame(X_pca_2,columns = ['PC1','PC2'])
plt.scatter(df_pca_2.PC1,df_pca_2.PC2)

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.