2
$\begingroup$

I have about 20-30 columns, all with different lengths. The first column has 25000 rows, the second column has 19000 rows, and it's different for all the columns.

All are the survey data with 0 (No),1(Yes), and 666 (for missing data) values. I want to perform PCA on these data. As each column is of unequal length, I am having a hard time doing PCA due to 'NaN' values for most of the columns that are shorter in length.

I don't want most of the information to be lost, so I am not willing to delete the rows of longer length columns and make every column equal in length.

Please advise me the possible solutions.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

According to my understanding PCA requires that you have the column of equal length, so you either need to

  • shorten the longer columns (basically just skip the incomplete observations) or

  • fill in the gaps in the shorter columns.

If you choose the second option, you'll need to learn about the concept of imputation (see the following link for reference Missing data imputation).

Please note, that you need to get rid of the dummy values representing missing response (666s) in your data before performig PCA, because these values are arbitrary (they also could be 9999 or -1) and would have significant impact on the results.

Furthermore, it's arguable whether PCA can be applied to binary data, see e.g. discussion under related SE question.

Improve this answer
$\endgroup$
3
  • $\begingroup$ @ainanov Thanks for the clear response. Also wanted to know, if I group some feature having same length (For example: if Feature 1 ,5, 17, 18 have same length of (29000), and feature (11,12,43,82,18) have same length of 20000), can I perform PCA separately and get the combined PCA afterwards? Maybe by combining eigenvectors, eigenvalues or some other way to get the combined PCA? Thanks $\endgroup$
    – Bishal Th.
    Commented Feb 15, 2021 at 18:33
  • $\begingroup$ @BishalTh. I‘m not aware of any workaround that could help. I suggest you’re adding this idea to your question so that more users could see it. Btw, it also would be helpful if you could amend your question by explaining what are you trying to achieve with PCA, maybe somebody can suggest better solution. $\endgroup$
    – aivanov
    Commented Feb 15, 2021 at 18:38
  • $\begingroup$ Yes, I can do that. Thanks for your time $\endgroup$
    – Bishal Th.
    Commented Feb 16, 2021 at 0:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.