I have a data set with 60000 rows and 32 columns. I want to use SVM (with some more constraints that make it more complicated)and I think 32 columns are too large. So I decided to use PCA. But when I load PCA, the first 20 component describe 85% of data which 20 variables still is too large I think, but it is better than 32.
I am wondering is that ok if I use PCA? when n>>m? if not what is a better option?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Why is 32 columns "large"? Even by (extremely sketchy) rules of thumb, you have a ratio of 1875 observations to one variable. That is more than reasonable IMO. What I would recommend: try PCA, keep how ever many components you think is reasonable, and validate your model properly. If there is improvement over not using PCA, go with PCA. You can even set the number of components to retain as a tuning parameter itself (pick number of components to whatever validates the best), or retain the PCA variables + original columns as well and fit a regular SVM on the combined dataset. $\endgroup$– aranglolCommented Jul 8, 2019 at 23:29
-
$\begingroup$ I would further advise that the cost parameter in your SVM is crucial and can (usually) adequately control the "complexity" of the fitted boundary. You may not need to do any feature extraction/selection; in fact it might make your model worse. $\endgroup$– aranglolCommented Jul 8, 2019 at 23:36
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
If you can, go for some non-linear dimensionality reduction technique. The most powerful are Autoencoders, but you can also use t-SNE or other manifold techniques.
The problem of PCA is that it can extract only latent factors that are linearly associated with your variables. Using non-linear techniques, less variables can let you capture more of the original variance.
