I recently ran a code to generate PCA for a movie ratings dataset. Actually there were two different datasets, a 'movies' and a 'ratings' one. The movie had about 9700 rows of different movie titles and genres. The ratings dataset had ratings of users for different movies. There were 610 unique users in the ratings dataset. I then merged the two datasets and then pivoted them to create the dataset for pca which contains userIDs as rows and all the different movies as columns. Predictably, the shape for this merged and pivoted dataset was 610 rows, 9700 columns. Here's the screenshot

enter image description here

After that, I ran this code to fit PCA for two components. Here's my 1st question, are two components fine to fit the pca when the original number of columns in 9700. Also, the below screenshot is of variance and singular values. Currently, I just see numbers but I'm not sure how much variability has actually been captured so can anyone explain how can make sense of these values. Below, you'll also see that I have transformed those values.

Variance Values

Finally, I plotted the values for the 1st and 2nd principal components on a scatter plot. I see points clustered together on the left side. What does that mean? And is PCA a good approach for dimensionality reduction for this dataset?

enter image description here

  • 2
    $\begingroup$ As an FYI you should post the code and data as formatted text, not screenshots. Also, include a link to the kaggle datasets if that's what you're using. $\endgroup$ Sep 20, 2021 at 1:34
  • $\begingroup$ Have posted this as screenshots because the code doesn't really matter in this case. I'm just trying to make sense of the output, the code is more of a reference. $\endgroup$
    – ShridharK
    Sep 20, 2021 at 1:42


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.