2
$\begingroup$

I am working on an exercise for using PCA for compression of images and I don't quite understand how to use it on the test data:

I have 300 images of hand drawn sixes, represented by 28x28 matrices, in the train data, and I have used PCA to find an appropriate low dimensional representation of these images (26 dimensions yields me the sought after 90% threshold), giving me a 300x26 Matrix, that I can use to project my images into that space

Now I have to test this with my test data of 10 similar images - so I have to project them into the same space.

Because I can't just use the train space (dimensions don't agree), if I understood correctly, I should run another PCA to find the Principle Components for these new images, but project them into the 26 dimensions as identified by my train PCA (PCA on just the test suggests that 6 dimensions would suffice, but I want the more accurate 26 from my wide range of training data)

But here is where I'm struggling: how do I centre the test data? To centralise the data before using PCA or the dual PCA, I deduct the mean from the data. When building my test PCA should I deduct the mean of the train data, or the mean of the test data?

Improve this question
$\endgroup$
6
  • $\begingroup$ Are you sure that the data from the train data is similar to that of test data? If they are not similar you cannot expect to have similar Components. By which you will be getting errors like Train and Test are not having same levels. $\endgroup$
    – Toros91
    Commented Nov 18, 2017 at 14:20
  • $\begingroup$ Hi - yes, images are all handwritten sixes, represented by numbers arranged in 28x28 matrices $\endgroup$
    – CrankMuffler
    Commented Nov 18, 2017 at 14:28
  • 1
    $\begingroup$ See PCA is used for dimensionality reduction. But you need to have same columns and similar values or else your PCA would give you different Principal Components $\endgroup$
    – Toros91
    Commented Nov 18, 2017 at 14:37
  • $\begingroup$ Yes, I have similar values - in this case I'm using it for compression $\endgroup$
    – CrankMuffler
    Commented Nov 18, 2017 at 14:38
  • $\begingroup$ But you will end up getting different components, as far as I remember it would be wrong if you compare but this is with my limited knowledge. $\endgroup$
    – Toros91
    Commented Nov 18, 2017 at 14:46

3 Answers 3

Reset to default
2
$\begingroup$

Found the answer in a different stackexchange question here: https://stats.stackexchange.com/questions/142216/zero-centering-the-testing-set-after-pca-on-the-training-set

Answer: Yes, zero center the test data with the mean from the train data

Improve this answer
$\endgroup$
3
  • $\begingroup$ yes, you need to do the same transformation as that of training and then try using PCA on test data. $\endgroup$
    – Toros91
    Commented Nov 18, 2017 at 18:41
  • $\begingroup$ Thanks - do you maybe have a good source I can quote on that? Been looking for one for a while $\endgroup$
    – CrankMuffler
    Commented Nov 18, 2017 at 18:42
  • $\begingroup$ Nope I dont have any source but if that works for your problem then thats great. I've used this technique for understanding the data for which I dint have any dictionary. So had to apply PCA to make components which make more sense than raw data. But i haven't used it in scenario like yours $\endgroup$
    – Toros91
    Commented Nov 18, 2017 at 18:45
0
$\begingroup$

When you run PCA on train data, you will get a vector space in lower dimension (After choosing the biggest components). As you know, this vector space contains the eigen vectors of the biggest eigen values. Now, using a proper function, project the test data into the vector space. So, you do not need to get PCA over test data anymore.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Yes, but I have the problem that I get 300 vectors with 26 features, one of each of my examples - but I have 10 examples in my test data, so I can't just multiply my new matrix into my data $\endgroup$
    – CrankMuffler
    Commented Nov 18, 2017 at 14:40
  • $\begingroup$ @CrankMuffler I don't get the problem. In any case, you can project the new vector into the new space by a dot product. $\endgroup$
    – OmG
    Commented Nov 18, 2017 at 17:23
-2
$\begingroup$

You need to combine the datasets first, run the PCA and then split the datasets afterward. Once this is done, you can train your model and then test it. The PCA needs to be done on the whole dataset in order to have the same components in both sets.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Sorry, but then what's the point of separation into train & test? $\endgroup$
    – CrankMuffler
    Commented Nov 18, 2017 at 14:36
  • $\begingroup$ If I understood correctly I use the train data once to get an accuracy - I can then use my results from that one time high complex problem to quicker solve my incoming test examples. $\endgroup$
    – CrankMuffler
    Commented Nov 18, 2017 at 14:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.