I'm rather new to PCA and was hoping to have some confusion cleared up. Lets say for example we have a feature matrix that's nx100 and I want to get it down to something a bit smaller, p-dimensions, without losing too much variance.
After applying PCA and receiving and new feature matrix nxp, I would use x_reduced to predict some target variable y.
My question is, after the transformation, the new reduced feature matrix has been rotated by the eigenvectors and is sitting on a new basis. Yet, our y has not changed relative to X_reduced.
I'm unsure about how y_original and x_reduced can be used for training since y has not changed with respect to x_reduced.
Is there a way to correct for this or am I not thinking about it correctly?