I am working on an exercise for using PCA for compression of images and I don't quite understand how to use it on the test data:

I have 300 images of hand drawn sixes, represented by 28x28 matrices, in the train data, and I have used PCA to find an appropriate low dimensional representation of these images (26 dimensions yields me the sought after 90% threshold), giving me a 300x26 Matrix, that I can use to project my images into that space

Now I have to test this with my test data of 10 similar images - so I have to project them into the same space.

Because I can't just use the train space (dimensions don't agree), if I understood correctly, I should run another PCA to find the Principle Components for these new images, but project them into the 26 dimensions as identified by my train PCA (PCA on just the test suggests that 6 dimensions would suffice, but I want the more accurate 26 from my wide range of training data)

But here is where I'm struggling: how do I centre the test data? To centralise the data before using PCA or the dual PCA, I deduct the mean from the data. When building my test PCA should I deduct the mean of the train data, or the mean of the test data?

  • $\begingroup$ Are you sure that the data from the train data is similar to that of test data? If they are not similar you cannot expect to have similar Components. By which you will be getting errors like Train and Test are not having same levels. $\endgroup$ – Toros91 Nov 18 '17 at 14:20
  • $\begingroup$ Hi - yes, images are all handwritten sixes, represented by numbers arranged in 28x28 matrices $\endgroup$ – CrankMuffler Nov 18 '17 at 14:28
  • 1
    $\begingroup$ See PCA is used for dimensionality reduction. But you need to have same columns and similar values or else your PCA would give you different Principal Components $\endgroup$ – Toros91 Nov 18 '17 at 14:37
  • $\begingroup$ Yes, I have similar values - in this case I'm using it for compression $\endgroup$ – CrankMuffler Nov 18 '17 at 14:38
  • $\begingroup$ But you will end up getting different components, as far as I remember it would be wrong if you compare but this is with my limited knowledge. $\endgroup$ – Toros91 Nov 18 '17 at 14:46

Found the answer in a different stackexchange question here: https://stats.stackexchange.com/questions/142216/zero-centering-the-testing-set-after-pca-on-the-training-set

Answer: Yes, zero center the test data with the mean from the train data

  • $\begingroup$ yes, you need to do the same transformation as that of training and then try using PCA on test data. $\endgroup$ – Toros91 Nov 18 '17 at 18:41
  • $\begingroup$ Thanks - do you maybe have a good source I can quote on that? Been looking for one for a while $\endgroup$ – CrankMuffler Nov 18 '17 at 18:42
  • $\begingroup$ Nope I dont have any source but if that works for your problem then thats great. I've used this technique for understanding the data for which I dint have any dictionary. So had to apply PCA to make components which make more sense than raw data. But i haven't used it in scenario like yours $\endgroup$ – Toros91 Nov 18 '17 at 18:45

When you run PCA on train data, you will get a vector space in lower dimension (After choosing the biggest components). As you know, this vector space contains the eigen vectors of the biggest eigen values. Now, using a proper function, project the test data into the vector space. So, you do not need to get PCA over test data anymore.

  • $\begingroup$ Yes, but I have the problem that I get 300 vectors with 26 features, one of each of my examples - but I have 10 examples in my test data, so I can't just multiply my new matrix into my data $\endgroup$ – CrankMuffler Nov 18 '17 at 14:40
  • $\begingroup$ @CrankMuffler I don't get the problem. In any case, you can project the new vector into the new space by a dot product. $\endgroup$ – OmG Nov 18 '17 at 17:23

You need to combine the datasets first, run the PCA and then split the datasets afterward. Once this is done, you can train your model and then test it. The PCA needs to be done on the whole dataset in order to have the same components in both sets.

  • $\begingroup$ Sorry, but then what's the point of separation into train & test? $\endgroup$ – CrankMuffler Nov 18 '17 at 14:36
  • $\begingroup$ If I understood correctly I use the train data once to get an accuracy - I can then use my results from that one time high complex problem to quicker solve my incoming test examples. $\endgroup$ – CrankMuffler Nov 18 '17 at 14:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.