Lets say I have a feature set of f0 to f1000. I am thinking of applying PCA on f500 to f1000 reducing their dimensionality. Can I combine this reduced set with the features f0 to f499 as the feature space for training a learning algorithm?

  • 2
    $\begingroup$ Just curious, for educational purposes, why you are interested in applying PCA to a feature subspace and how you choose which ones? I think it is useful to share your experience and edit your question such that other learn. $\endgroup$ – TwinPenguins Feb 7 '18 at 10:59
  • $\begingroup$ The subset of features in my problem, from f500 to f1000 are sparse and are of Boolean type. Hence I am interested in trying out PCA to find a reduced set. $\endgroup$ – lone_wolf13 Feb 7 '18 at 18:26

Yes, absolutely. Simply split your data into two sets feature-wise, apply PCA to one of them, and then stick them back together again. How to actually perform this will vary depending on your programming language/frameworks, but it is trivially easy in python + pandas, for example.

  • $\begingroup$ At what stage should I apply the StandardScaler if I am doing the above pls? I have a set of features which are client related features where I do not wish to apply PCA and then I have market related features (which is admittedly very large) and i would only like to apply PCA on them. I believe I will need to apply StandardScaler on the market features anyways before the PCA. Should I separately apply StandardScaler on the client features and then append the 2 for final DF? Thanks! $\endgroup$ – spiff Oct 9 '18 at 4:20
  • $\begingroup$ You should definitely apply the scaler before PCA, as it can give poor results when operating on data where each variable has differing scales. If you don't want to apply standardisation to the client features, then that's fine; just split out the market features, scale them, and apply PCA before appending the transformed data to the original DF. By the way, it may be worth taking more than two principal components! $\endgroup$ – timleathart Oct 9 '18 at 10:59
  • $\begingroup$ oh yes, I have 128 features in the 2nd set - will try and take all features which explain atleast 70% of the variance. Thank you very much! $\endgroup$ – spiff Oct 18 '18 at 6:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.