I am learning about dimensionality reduction and I understood that one of the most used techniques in ML is PCA.
If I understood correctly, I use PCA whenever I want to reduce the number of features which should be mostly linearly separable (independent ?).
When the features are not-linearly separable (linearly not independent?), a nonlinear technique is required to reduce the dimensionality of a dataset and therefore I use KernelPCA.
Question: Supposing that what I just wrote is correct, how can I know in advance if the features are linearly or non-linearly separable before i decide which technique to use? So far the only way I was able to "guess" is by plotting the features after a PCA and checking if i can separate the new features through straight lines/surfaces. If I am not able to do so, then I apply Kernel PCA. Is this approach even correct?
- Feel free to modify my question where the "?" are :)
- My features don't have labels. It's an unsupervised problem.